Sometimes, you don’t need to measure it to manage it: metrics for Millennials

By November 30, 2016Uncategorized
advertising agency blog

A couple of weeks back Carrots, Sticks, and Sticky Carrots guesstimated the percentage of your marketing budget your social spending shouldn’t dip below. Next, The Winter of our (dis)content looked at how different channels appeal to different parts of our brains, with a nod to psychologist Dan Kahneman’s work on System 1 and System 2.

This time around, let’s add to that duo—with an idea for measuring the touchier-feelier metrics consumer engagement is moving towards. Since it describes most of the people in our office, we’ll recruit every marketer’s fave demographic as our lab rats: the Millennials.

Much ado about Millennials

First, let’s burst a few bubbles. The Millennials—people born since 1980—aren’t a new paradigm, or resistant to research, or too weird for words. (Unless you’re in Shoreditch of a Friday night.) They’re not even that different from their parents in Generation X; the social shift between, say, the prewar generation and their offspring the Boomers was far greater.

And they’re not “up-and-coming” any more. In the last year or two, Millennials became the largest segment of the workforce. That makes them mainstream.

But since they came of age just as the world was connecting up, they do have some traits that differentiate them from their parents. Specifically, in how they acquire information; how they build brand preferences; and how they communicate their fondness (or dislike) of a brand to their peers. Which contains an idea worth exploring. Should we be measuring the individual’s propensity to click or buy . . .  or the peer group influence that drove her to do so?

Back, and to the left

Such second-order metrics are derivative: they involve looking “across and back” the dataset. Not just at the string of data points that describes one consumer, but at the surrounding  web of data that pushes and pulls her this way and that. Sounds complicated? Not really. It can be summed up in something you’ll all recognise: the Net Promoter Score.

Net Promoter: it’s just sooooo ultimate

The NPS is the gold-standard metric for many customer experience programmes in the world’s largest companies. The pointy bit is the reader’s answer to would you recommend us to your friends and family? (What its creator calls the Ultimate Question.)

It’s powerful because it asks the consumer to consider how other people would feel about the product or service. And not just *any* other people, but the people who know him best, that feature most prominently in his life. That makes the Ultimate Question a derivative metric, one that connects up a much bigger web of data points than most metrics measure.

Best of all, it then goes one better. It turns that slippery qualitative animal, “how people feel”, into a quantitative output that’s easy to use in statistical modelling . . .  a simple score from 0 to promoter score ad agency

Consumers scoring your product or service 0 to 6 are Detractors: they aren’t likely to recommend you. (So it’s your job to persuade them.) 7s and 8s are Passives: they like you, but don’t shout your name from the rooftops. 9s and 10s, however, are your Promoters. They buy your product or service, and persuade people they know to buy it too!

Promoters are your best friends in all consumerdom, and it’s worth spending time and money to nurture them.

Here’s the twist. NPS is already among the world’s most widely used CSAT (customer satisfaction) metrics. But the NPS itself is a catch-all. It’s simply the percentage of your Promoters minus the percentage of your Detractors, to give a single number. And we feel that’s one step too far.

Maybe we don’t need to measure NPS, to manage it.

The trouble with NPS

Because when you’re interested in consumers as individuals, as all fmcg brands should be when marketing to Millennials, the final number matters less than the data that led to it.

So it’s worthwhile looking not at the final NPS score in isolation, but at the dataset: those answers to the Ultimate Question, in all their variety. Because that dataset is multidimensional. It hasn’t lost all its insights and nuances by being agglomerated into one number.

This dataset tell us how each individual consumer feels about you, when considering the judgement of his peers. Which is a brilliant way to measure peer group inflence on a buying decision. So while CSAT experts tell us NPS is the ultimate answer, we care more (with apologies to the late Douglas Adams) about the ultimate question.

In the age of the Millennials, could NPS be taking on a new life—as the only marketing metric that really matters when marketing consumer products?

Let’s do some Scorch-style back-of-envelope research into this, with a technique called Structural Equation Modelling, or SEM. Don’t worry: like most statistical methods, it’s nowhere near as complex as it sounds. (OK, we lie. It is complex. But it’s worth it.)

SEM: telling it how it is

SEM isn’t one statistical method, but a family of them. (The regression analysis you saw in Sticky Carrots is an SEM method.) What these methods have in common: they let you forecast stuff. Any two columns of numbers will correlate in some way, but SEM tells you if variables have a predictive effect. Not “Does X relate to Y”, but “Does X lead to Y?”

This means SEM does more than tell us whether there’s a relationship between a company’s NPS score and its audience’s connections across social media. What we want to know is whether you can drive an uptick in NPS by driving Likes and Shares—i.e. does social media activity predict an increased NPS, or just correlate with it? And if so, what matters?

The regression graphs from Part 1 used simple linear regression: one dependent variable, one independent variable. But to look at webs of data with multiple connections between the data points, we need to up our game: multiple regression, where what we’re trying to predict depends (hence “dependent variable”) on more than one indie.

Let’s look at multiple regression graphs for two gender-biased fmcg brands: Proctor & Gamble’s Gillette and Unilever’s Dove, by looking at some simple numbers from their Facebook pages. Why Facebook? It’s because of a recent feature: the choices that appear when you “Like” a Post. Comparing numbers of Likes, Loves, Sads, and Angrys give us a useful proxy for how people are feeling—one that fits nicely with the Detractor, Passive, and Promoter categories of the NPS model.

Dove: soothing consumer pain

Our methodology keeps it simple. (We like simple.) Dove’s NPS score at the end of 2013 was 53. So to build our model, we collected a set of social media over 2013 (Facebook didn’t have Loves, Sads, or Angry options on posts back then) and normalised it for the dependent variable, i.e. 53.dove social agency

Next we collected another set from 2016, and applied the same normalisation – basically, relating the NPS number with whether the trend is rising or falling. (On the basis that if looking at social media has any predictive effect, looking at the equivalent data during 2016 will produce Dove’s Christmas NPS for 2016.) Figures for Q4 2016, which hasn’t finished yet, were estimated.dove digital agency

As you can see, the trend is slightly up, with a big bulge in Likes in the first half of the year. Delving into the data – looking at ratios of Posts to Likes and other gubbins – predicts Dove will end 2016 with an NPS of 54. Staying at the top of its game. Which, if you know this brand well, feels about right.

Here’s what makes the 2016 figures useful. Dove doesn’t do as much as you’d think on the world’s largest social network; a typical Post gets around 400 interactions. But about 3% are Loves not Likes, and 1% are Sads or Angrys, indicating Detractor-like behaviour. 3% of readers also Share each Post – and the ratios in which they do so contribute greatly to our insights. If people are reacting strongly, it indicates stronger involvement . . . perhaps these “extreme” Likers are our NPS Promoters?

Checking into web sources, the media seem to agree, believing Dove’s customer equity has stayed steady in the last two years. So it seems, yes, strength of sentiment (in either direction) does have some predictive power on a brand’s NPS. What seems to matter: engage with and look after your strongest fans . . .  and your biggest critics.

Gillette: the unkindest cut

Gillette’s published NPS score (from 2013) was a lowly 28. In this connected world, nobody really loves Gillette enough to shout its name. What’s in store in 2016 for the razormaker?

gillette digital agency

In the data is our answer. Gillette has more overall activity that Dove – often with per-Post viewership in the thousands—but as a proportion, far fewer readers Like each Post; mostly, nobody Loves a Post at all. And fewer than 1% of a Post’s audience Share it. Irrespective of how many readers there are, they’re not deeply engaged. Could this have predicted that awful NPS figure?

Translating the graph for 2016:gillette digital social agency

Again, there’s plenty of activity, but Loves, Sads, and Angries were so low we had to remove them altogether; they made the graph unreadable. The predictive model forecasts an NPS by Christmas of a lowly 22! Customers aren’t exactly raving: that grimace when they run a pack of FusionGlides over the checkout scanner may be turning Passives into Detractors and stopping Promoters ever emerging.

A check with the press again confirms this prediction: pundits are cool on Gillette, contrasting its high-cost product with newcomers like Dollar Shave Club.

Does our hypothesis hold: a company’s NPS depends not on overall audience, but on the few who really engage?

Cross-checking on Twitter

This is a thin dataset, so we did some cross-checking with Twitter. Here, the viewership figures are reversed: Dove Tweets a lot, Gillette far less. But the conclusions are the same. With over 16,000 Tweets, Gillette gets only a handful of retweets—a proxy for recommending—barely a twentieth of Dove’s.

The social media teams obviously focus on different channels, but the broad outcomes for each are the same: your NPS score depends on how many cheerleaders you’ve got, not the size of your audience.


So there you have it. Consumer advocacy—Sharing and Loving—does seem to predict a brand’s overall NPS score. But it’s got to be the right kind of advocacy. Gillette is cutting itself to ribbons; Dove is lathering up nicely. The conclusion: don’t measure your clicks and pageviews—look for your Promoters among the Shares, Likes, Loves, Sads, and Angries.

As usual, here’s our disclaimer: the research above is from a small dataset with disparate sources. But it’s an interesting thought: that turning just 2-4% of Likers into Lovers ripples out into big dividends for your entire brand. 

We’d like to explore more ideas like this with you. Why not get in touch with Scorch London for a chat?