The Quantum Mousetrap

Mark Eduljee's blog about Social Media Insights Intelligence and his FlightSim Movies

The siren call of Sentiment Analysis

June 1st, 2014

In days of yore, it was said that mariners seeking direction and safe passage, would be lured to all manner of disaster by a song of promise which was wonderful to listen to, captivating, beautiful, simple, and appealed to their most basic human emotions.

The siren song of sentiment analysis and data

In today’s modern business voyage, where there is a similar need and search for safe passage, direction and smooth sailing, corporate captains can be observed urging their organizations to tune into the siren song of sentiment measurements to act as their compass, motivator, resource allocator and catalyst for decision making.

Why is this? It’s because the basic expressions inherent to the Sentiment analytics output models are easy to understand and frame in a conversation. Sentiment measurements generally roll up to 3 simple states: Positive, Neutral, and Negative. In more layman’s terms: Happy, Sad, Meh! Everyone gets it; they are emotional states that are easy to relate to. It’s Simple Elegant Measurable. It calls.

And from there its but a small leap towards group-think, sound-bite solutions: How often have you thought of, or heard some version of this get-quickly-to-solution refrain?…“If only I had a scorecard that showed me my “sad” number, their regions, and themes, then, all I need to do is to figure out what went wrong, address, and that would convert them to.. !! Happy!!”

That sounds like a reasonable plan and process, and seems logical. You get the cookie for “connecting the dots”. Kudos for strategic thinking! This will help the company and others around you get better. Direction and safe passage! Smooth sailing!! Promotion here I come!!!!

Umm…“Captain, my captain…cover ye ears and drown yon evil siren sentiment song, for there be all manner of foul, confidence-busting rocks and shoals of confusion-seaweed out tharr!

But before we go further, this is NOT about ignoring or dismissing sentiment data. For any business decision maker, the need to understand customer emotion about the brand, its experiences, engagement, or communications, IS important. Like any data, sentiment is but one arrow in a quiver of analysis options that can be useful when used with intention and a full understanding of what it should or not be used for, its limitations, and of how it can easily be misinterpreted or misused (and so become dangerous for their ship and crew).

The simplicity of its 3-state data is in fact the root-cause and reason for Sentiment data overuse and abuse and these are sure signs to look for that point to Sentiment data not being used appropriately: …when sentiment measures becomes the preferred basis for decision making
…when it is adopted as a key business and performance success measurement
…when its numbers are used to determine budget allocations
…when it is the dominant trigger for marketing, communications, or customer engagements
…when service contracts are awarded based on how well, easily and/or quickly sentiment can be measured by those services
these all sure signs that there be rocks and propeller-fouling seaweed ahead.

Look around your business or where you work. Do you feel there is a disproportionate reliance on this simple measurement?  If yes, then this follow up question may also be well worth asking: Fundamentally, has this sentiment measurement strategy helped the business over the LONG term? Or, is it resulting in scorecards causing short term and reactive operational, communications and engagement activity (and churn)?

If sentiment were to be used as a long term value-add for the good ship Enterprise, it MUST meet the following data requirements: (actually this is true for most data). Any BI or Insights (what’s the difference?) measurements must be:
1) Meaningful – provides a level of detail enough that it can be assigned to someone who is holding themselves accountable to address
2) Relevant – information that has some bearing on what drives the business, its priorities, and direction
3) Contextual – information that can be used to influence and inform choices and outcomes by being repeatable, reproducible and with samples and examples that can be tested and documented
4) Timely – occurring when the opportunity for maximum gain is evident

Now I know what you are thinking…“Wait! What about the requirement that the data and insights be ACTIONABLE? Data needs to be actionable otherwise its useless! Why is that missing from your list of requirements?

You are right in one respect: Often the term Actionable is listed as a key data requirement.

But, in practical terms however, actionability is really a function of an enterprise’s ability and desire to take action, and that is defined and enabled by how it is led, managed, funded and organized, which, then deterimes whether taking action is an embeded culture, and how costly that process will be…in other words actionability a is a desired business state, or customer RESULT, not a data requirement. In truth taking action is a business objective whose probability of happening is based on, and is directly correlated with:
a) the methodology, framework, and processes in place and used to acquire and process source data
b) data confidence and quality generated by BI, Insights, Monitoring, or Listening programs and systems (read about the difference between Listening and Monitoring, and BI and Insights)

Therefore, once the 4 requirements and 2 objectives mentioned above are met, then the data becomes actionable.
Get them right, and the data becomes, by virtue of those foundational conditions being true, “actionable”.

Meeting the requirements and objectives listed above to make Sentiment data actionable is not a trivial exercise.

While there are numerous tools and services who say they meet the requirements, it’s all a bit of smoke and mirrors. Admittedly, while some (and these tend to be the more expensive options) have fairly sophisticated entity, semantic, and language processing science and IP underlying their charts, graphs and other eye-candy sentiment analysis delivery, the majority of these services provide questionable sentiment numbers, at best. Caveat emptor!

But even with caveats and context, no matter how great the tools say they are, there remain the following fundamental truisms about sentiment that the business must own and manage…because, at its very core, at its very essence, “sentiment”, as a long-term business value-add measurement suffers from the following:

Like the Sirens, Sentiment is beautifully ethereal. In fact it’s a reflection of a current state of mind, and does not reflect or have ANY connection to future potential or status. For this reason, it should only be used to reactively address very short term situations or very tightly scoped scenarios.

Like the complex emotions it is meant to measure, Sentiment analysis is not simple. Yet, in an attempt to make it so to drive wider adoption it has been reduced to a 3 point measurement that cleverly includes the catchall: Neutral. That is why the neutral bucket tends to dominate most sentiment results. And it is also why Neutral is generally treated with respect, but also ignored. “If they are not against us, then they must be sort-of with us…Good to go!!” Do you know of anyone who has a performance commitment that requires their work to reduce Neutral and shift those numbers towards Positive? Hmm…

Like the richly varied stories about Sirens told by sailor to sailor, Sentiment measurements are based on storied standards that vary from vendor to vendor…each claiming that their product offers the most accurate results. But where are the industry-accepted sentiment standards? There are none. Every company uses their own model. Generate sentiment scores on the same data from 5 different social media tools and you’ll get 5 different results — all claiming to be right.

Like Sirens who bend and influence minds to follow their call, so to can those providing Sentiment ratings be biased towards a mindset that effects ratings. There are any numbers of ways to provoke specific emotions that would influence sentiment scores for specific data sets. And because Sentiment is based on, and is a measurement of a shifting emotional state of the mind (that is in turn influenced and effected by millions of unknown, interconnected variables like culture, age, expertise, maturity…), it is relatively easy to then use tactics (often deployed unintentionally) to influenced the framework and mechanisms which records the data (or choose the source) to be used in sentiment analysis into getting what is wanted (as opposed to what’s really needed).  As examples…“provide feedback” sentiment scoring by its very nature will generally be neutral to negatively framed because it’s about improving what perceived to be broken (but paradoxically, it’s provided in the hope for a positive desired outcome – so where does that factor into the equation?), or, if the same feedback ask was framed as being about “opportunities to improve” will shift towards using more positive language. Also the timing of when a sentiment gathering instrument is presented to gather data, or its labeling can and will influence the sentiment a person feels as they contribute data …“send a smile”, “show how much you care”, “talk to us – you are important!”…are all examples of ways sentiment can be influenced to drive results wanted (not needed)

Like the Sirens of yore, Sentiment is an emotive phenomenon — there, but not there: not much different from rainbows or mirages that seem real under the right conditions and are characterized by shifting states with no anchors. At its core, Sentiment measurements are an interpretive science and not based on scientific modeling. Don’t agree? Here’s a simple test: Give me a sentiment score and its driver weight that, if acted on would move the sentiment/emotion being measured by that driver weight.  You could do this through with Satisfaction, or affinity or loyalty measurements using driver regression and dependent variable driver modelling and analysis. Not with Sentiment. So what this means is that in the end sentiment is a number that gets pulled out of a hat, and is determined by which hat it’s pulled from.

Having a measurement that shifts and changes to reflect real time market or competitive-driven emotion can be enervating and frustrating, and demoralizing for the enterprise struggling to do the right thing with finite resources. Picture facing a bewildered VP who has just been told that the 3 things he (or she) is spending millions on to change based on last week’s sentiment analysis has had no effect on negative sentiment scores because customers have shifted the focus of their emotions this week to some other shiny object. It’s the surest way to lose credibility and a seat at the table.

Of all the issues with Sentiment, the lack of standards stands out as THE most troubling, and a prime reason to go to it as a last resort. Unfortunately it’s this lack of accountability to any standard that is most manipulated to achieve self-serving goals both within the enterprise and by vendors and consultants.

Having BI and Insights systems (different) which deliver high confidence information, that result in actionable data, derived from predictable and repeatable methodology which can be explained and justified, will drive moral, a sense of accomplishment and eventually, organizational and business health. Sentiment is a siren’s call in that regard.

This is why I steer my stakeholders away from Sentiment as a success measure, and instead work closely with them to seek more stable, higher confidence data in its place that is more suited for the business need rather than the knee-jerk emotional want. (More about the differences between needs vs wants here) 

So what’s the alternative??

First: Establish WHY Sentiment analysis is being asked for. Sentiment is a useful analytics option for any data science team to use IF the business need is short-term and reactionary… efforts like PR, monitoring or reacting to an event, campaign, opening, or an evolving/dramatic human-interest story, or, keeping tabs on services operations and availability. Sentiment monitoring (that’s all it really is) has its place when used with intention and forethought about its scope and limitations. Use it wisely.

Second: By all means, feel free to be obsessed with the latest shiny sentiment trend visualization, but balance that with responsible, longer term, analytics investing for more stable, actionable Insights and BI. Invest in the people, tools, and process to build frameworks that are necessary to provide standards-driven results that return meaningful, relevant, contextual, and timely data. Only then will action-taking for permanent customer experience changes become real.

Third: Embrace and accept the idea that Big data and Insights intelligence analytics is neither simple, quick, cheap, or easy. If you absolutely must have sentiment analysis, set expectations and clearly come to terms with its scope and limitations. Identifying, reporting, and using high confidence data and insights that drive action and change requires investment, specialized skills, a long term horizon, sponsorship, focus, discipline, time, and leadership. 

In other words: snap out of it!! ...and watch out for the songs from the Sentiment sirens! If you do, your business, customers and people will thank you.




Image credit:


Insights Intelligence: the untapped gold mine hiding in plain view

May 17th, 2014

Here’s a key question to ask any business: Does it know what customers are saying about its products and services?
Often an answer to the question will not be framed in terms of the highest priority top 10 experience improvements, change opportunities and requirements, but rather in terms of activity and effort measurements.

This is done using BI scorecards and dashboards which illustrate business activity and effort, examples like: operations status, forecasts, trends, with associated metadata — how much customers liked you, retweeted, pinned, or shared us. The answer is usually: BIG DATA!! Collect it all. More is King. Activity trumps taking action.


It’s important to ask the key question for 3 main reasons:
1) Every business should have a clear understanding of its customer experience…not what the biz thinks it is, but based on the voice of its customers (what they think, do, say).
2) It’s not enough to just have quantity measurements, like, likes, or SAT, sentiment and retweets. Or recommendations. It’s more important to know why that’s NOT happening.
3) Numbers and scorecards without root-cause analysis can be interesting yet hard understand, and while directional are open to biased interpretation.

To be clear, understanding the impact of business, marketing, communication, support effort and activity is important, and necessary. But that’s Business Intelligence (BI), not Insights Intelligence (InI).

One is not better, or less needed than the other. BI and InI are just 2 sides of the intelligence framework. They are interconnected, yet distinct; unique yet joined. It is this yin-yang nature of Business and Insights Intelligence which causes confusion and leads to Insights Intelligence being hidden, or worse, (unintentionally) ignored.

BI, Big data, Data science, Listening – these overused buzz words are all the rage, but they represent only one side of the Intelligence equation. Let’s hit pause here for a minute. Let’s look at the other side…

The qualitative Yin to BI’s quantitative Yang, is customer voice Insight Intelligence.


What is Insights intelligence?  It is actionable intelligence about the customer experience derived from the digital imprint and breadcrumbs of your customers’ voice that is available in unstructured text data that humans see as social posts, survey “reasons for your ratings” questions, in product and experience reviews, news, forums, blogs and comments, in all languages, across every region/geo, of every second of every day.  

It is the global voice of your customer across the customer journey. And it’s all there for enterprises that want to listen and learn from it.

Herein lies the biggest, untapped, most underutilized, and probably the most misunderstood and ignored customer voice goldmine. It’s not easy to get. It is VERY valuable when gotten.

Now I know what you are thinking – this is not the flip side of BI, this is BI!  Umm…No it’s not. Because, all data is not BI. It’s like saying all water is liquid, or all Roses are red.

It’s important be thoughtful and nuanced about intelligence types, for, while all of this intelligence is about the business, lumping all types of business or customer intelligence into the generic and overused “BI” bucket drives certain source collection, analysis methods, resource priorities, processing strategies, reporting and analysis behavior in ways that is antithetical to understanding the “why” behind an experience – which is what Insights Intelligence is all about.

Having great effort-measuring BI is important. There is just a different path to acquiring Insights intelligence from voice of customer analytics.

Consider this: In the old days (a couple of years ago), customers had to make a special effort to visit a company walled garden to give feedback, ask a question, seek a recommendation, get help, or information. . Now?…no more is it a prerequisite to go to any fixed location to do any of that…conversations about a company’s brand, operations, competition or experience are now borderless and boundless.  Conversation happen in places where customers wish to have them, when they want to have them, and with who they want to have them, with or without the company being present.

Welcome to the complete democratization of customer experience conversations on the global stage.

Imagine what this means for you or your company: hundreds of millions of data points all about what’s needed, what’s important, what customers want or don’t want, all given willingly, begging you to hear what’s important, and documenting for you what their product, device, or service experience was. Always on, always relevant.

Actually, forget the key question I asked above. Here’s a better question: Does the business have the Insights Intelligence leadership in place with right level of resources, tools, skills and processes in place to hear this global voice; to process and prioritize the Insights, and to use the underlying customer context to inform and build winning product, service, and communication and engagement strategies?

And more importantly: Is the business organized in a manner to ACT on the Insights Intelligence in an effecient and timely manner?


Next time: The steps towards building an Insights Intelligence and listening framework.


IMAGE CREDIT: Cartoon of the 2 office workers: Chris Slane