The Pet Rock. That is the first thing that comes to mind when I hear
people talk about Net Promoter (NPS). OK, if you aren’t old enough to
know about the pet rock, check it out at wikipedia, http://en.wikipedia.org/wiki/Pet_rock , better yet, ask your Mom or Dad.
So, what do the Pet Rock and NPS have in common? They are both fads that have no value.
Pretty strong statement considering all the press it gets,
considering all the fantastic claims its creator makes. While I could
go on for days about all the problems, let’s just focus on a couple of
highlilghts (or maybe lowlights is a better way to describe them).
First myth of NPS: NPS predicts revenue growth.
Time for stats 101. At best Reichheld tries to show correlation between
NPS and revenue growth. Correlation and causation are very different.
Causation predicts the future, correlation tries to explain the past,
but often confuses coincidence with relationship. That is the first
problem. How about an example. Did you know that the amount of damage
caused by a fire is highly correlated to the number of firemen fighting
the fire? But, the amount of firemen fighting the fire does not cause
more damage. Hmmm. So what is the cause. It is missing from the
equation. If we introduce the size of the fire, we see that the size of
the fire causes more firemen to fight the fire and the size of the fire
also causes more damage. That is causation. So, let’s bring that back
to NPS. Customer Satisfaction causes revenue growth and customer
satisfaction causes the number of recommendations to increase. That is
the causal and predictive relationship we should care about.
The second problem is even bigger. Take a close look at the data that
is presented to demonstrate this correlation. It compares revenue
growth over a period of time, and then looks at NPS at the tail end of
the period (pages 192-194 of the book). For example, comparing ISP
providers three-year growth from 1999-2002 and NPS scores from 2002.
Now, if we want to take the leap that correlation could be interpreted
as causal, we would add a time dimension. Something that happens today
being correlated to something happening in the future. It is a leap of
faith, but at best, that is what the data presented could show us. The
data presented in the book and papers would actually suggest that
revenue growth drives NPS. Is that possible? Well, let’s look at the
psychology of recommendations and human nature. When we make a
recommendation it reflects back on us. So, we are inclined to recommend
those things that are popular (experiencing revenue growth), even if we
haven’t experienced them. That explains the possibility that revenue
growth drives NPS. The exact opposite of the NPS claims. The real issue
here is, when you have a theory, use data to prove it. Bottom line, I
have not seen that data for NPS.
Now for my pet peeve (I guess that goes with the theme of the Pet
Rock). All those people that should know better, but are jumping on the
NPS bandwagon. You know who you are! Many don’t know better (ignorance
can be bliss), but many are jumping on the bandwagon because they can
capitalize on it, and monetize it. You should be ashamed of yourself.
It is time you gave back your pet rock…and got back to reality.
Remember those key principles we learned in school. Things like the
scientific method. Reichheld himself states in his blog “Frankly, we
see little value in continued debate about cause versus correlation,
timeframes, or statistical methods”. I guess we should throw out the
hundreds of years of great research. Who knows, maybe the world is
really flat.
Check back for the second myth.
Fabulous piece! This will be a key reference for future discussions.
I see NPS being used more and more. I have used it with my clients with limited results. The best use of it is to ask the follow-up questions (”what was the reason for your score?”) and segment those answers by Promoters and Detractors. This will at least give you insight into why customers would OR would not recommend you. Great post Larry!
[…] See Larry Freed’s blog for more discussion on this study. […]