FREEDyourMind

Welcome to the Bandwagon

As some of you may know, I am not a big fan of the concept of Net Promoter (sometimes referred to as NPS). While measuring positive word of mouth (likelihood to recommend) and negative word of mouth are very valuable and important metrics, the way the NPS score is calculated causes the “one number you need” to lose all precision and reliability–in the best case making it a meaningless number and in the worst case causing it to be misleading, potentially driving you to make BAD decisions. You know the old saying, garbage in - garbage out.

Welcome to the bandwagon of market research sanity - Timothy Keiningham and colleagues. They recently had a paper published in the Journal of Marketing (July 2007) titled, “A Longitudinal Examination of Net Promoter and Firm Revenue Growth.” Let me share with you a few quotes from the paper.

“The clear implication is that managers have adopted the Net Promoter metric for tracking growth on the basis of the belief that solid science underpins the findings and that it is superior to other metrics. However, our research suggests that such presumptions are erroneous. the consequences are the potential misallocation of resources as a function of erroneous strategies guided by Net Promoter on firm performance, company value and shareholder wealth.”

So, lets put that in simple terms: Net Promoter doesn’t work!

Another important quote

“This means that Net Promoter was tied to past growth rates (as opposed to future growth rates). As a result, the data do not help identify whether Net Promoter levels are linked to current changes in revenue growth.”

To clarify – while trying to substantiate his claims on Net Promoter, Reichheld fails Statistics 101. Now I know that he has claimed that executives don’t care about statistics, but I hope they care about accuracy. The data used to support the Net Promoter theory compares revenue growth in a prior period to Net Promoter results in a following period. That would imply, in the best case, that revenue growth drives net promoter. Hmmm… that’s not what Reichheld says.

Let’s look a little further. In the book promoting Net Promoter the biggest case study referenced is Enterprise Rental Car. Believe it or not, they do not ask the questions prescribed by Reichheld, how likely are you to recommend. Instead they ask how satisfied are you. But Reichheld takes the leap to say in his book, and I quote:

“These questions struck pay dirt: the one question at the top of the page: ‘Were you completely satisfied?’ accounted for a startling 86 percent of the variation in customer referrals and repurchases. Those who gave the company a perfect 5 on a 5-point scale - the equivalent of promoter-were three more times likely to return to Enterprise than a customer giving a lower score.”

To put in simple terms - Reichheld “tricks” the readers into thinking Net Promoter works - and it doesn’t. His most profound case study doesn’t even use the “ultimate question.” A true embarrassment to all of those that have drunk the Net Promoter cool-aid.

So welcome to the bandwagon of Market Research Sanity - Timothy Keiningham and colleagues.

  Comments

1.    
         Paula Thornton      
          
         June 27th, 2007      |       5:09 pm      
      

   Fabulous piece! This will be a key reference for future discussions.

        
2.    
         Paul Schwartz      
          
         July 3rd, 2007      |       7:40 pm      
      

   I see NPS being used more and more. I have used it with my clients with limited results. The best use of it is to ask the follow-up questions (”what was the reason for your score?”) and segment those answers by Promoters and Detractors. This will at least give you insight into why customers would OR would not recommend you. Great post Larry!

        
3.    
         The Great Customer Advocacy Hoax      
          
         July 28th, 2007      |       1:56 pm      
      

   […] See Larry Freed’s blog for more discussion on this study. […]

Are You Satisfied With E-Gov?

When was the last time you were in pursuit of information from government? When was the last time you had to fill out a government form? The internet has played a major role in making that process easier. It is hard to remember a time that e-gov wasn’t available to us to check on the status of a tax refund, or find out how to renew your passport, and the list goes on. But the last time you visited a .gov site, was it a good experience? Was it all it could be? Are you going to return to the site in the future? What could that .gov site done better?

Those are the burning questions on the minds of those responsible for the e-gov efforts. The most recent report on Federal Government Website Customer Satisfaction was recently released by The University of Michigan’s National Quality Research Center and sponsored by ForeSee Results.

Citizen satisfaction with federal websites registered a slight improvement, according to the second quarter report from the University of Michigan’s American Customer Satisfaction Index (ACSI) E-Government Satisfaction Index. The e-commerce and transactions category showed the greatest increase of any other category, inching the E-Government Satisfaction Index forward 0.4% to 73.7 on a 100-point scale and reversing last quarter’s drop.

Although aggregate satisfaction with federal websites improved incrementally, it still lags the private sector, which continues to set the bar for online satisfaction. E-government scored 8.5% and 3.8% behind private sector e-commerce (80) and e-business industries (76.5), respectively.

Private sector e-commerce is one of the strongest performing sectors measured by the ACSI and still holds the edge over e-government transactional and e-commerce sites. But as more federal websites allow citizens to do business online with government, satisfaction is improving. On aggregate, the customer satisfaction score for sites that offer transactional or e-commerce capabilities rose 3.5% from last quarter to 76.8.

Three of the strongest performing e-commerce and transactional sites belong to the Social Security Administration (SSA). Internet Social Security Benefits Application site (88), Help with Medicare Prescription Drug Plan Costs site (87), and Social Security Business Services Online (83), provide superior satisfaction by even private sector standards.

The Top 100 E-Retailers

But before we get into that, help us fight cancer on June 9th and  10th

This year, my wife and I have decided to participate in the West Bloomfield Relay for Life. The Relay is an overnight event that celebrates survivors, remembers those that lost their lives to cancer and raises money for the fight against cancer. Funds raised during the event go to cancer research and patient services such as transportation to and from treatments and respite care.I also hope you’ll consider helping us support this great cause. Checks can be made payable to the “American Cancer Society” and sent to me at 4330 Windridge Ct., West Bloomfield, MI 48323. To make a contribution on-line using a credit card, please visit our Relay website using the link below.Thank you for your help and support of our efforts to help find a  cure for cancer.

Larry

Follow  This Link to visit my personal web page and help me in my efforts to support  Relay For Life of West Bloomfield

http://www.acsevents.org/relay/mi/westbloomfield/thefreeds?faf=1&e=1081995739

Now to the Top 100

What is the Top 100? Working with Internet Retailer to identify the top 100 Online Retailers based on online sales, we measured satisfaction of visitors to those retail sites. We used the proven technology that drives the American Customer Satisfaction Index. It is an important point that we measured site visitors, not only purchasers. This is important for a couple of reasons. First, many visitors to online sites preference is not to purchase online, but to purchase through other channels. We don’t want to ignore the value that the site of an online retailer can play, beyond a commerce tool, but also as a marketing tool, a research tool and a support tool. Secondly, focusing on visitors often presents the best opportunity to impact the bottom line of a retailer. If you are converting 5%, that means there are 95% that are visiting your site and not purchasing.

Why using the American Customer Satisfaction Index (ACSI)?  There is no better measurement technology. It is the only technology with the financial evidence behind it that it works. Evidence that stands up to the scrutiny of the academic and marketing communities. The ACSI gives us credibility, accuracy, reliability, precision, predictability and actionability. No other methodology comes close. While there are some copycats out there, none have the scientific foundation of the ACSI and none have the evidence that it works.
Why measure satisfaction? While it is important to understand the financial and behavioral metrics that we are used to (sales, conversion, visitors, etc.), those tell us what happened in the past. Satisfaction tells us what will happen in the future. If 1,000 visitors came to your site today and bought a certain product, what does that tell you? It tells you that 1,000 visitors came to your site today and bought that product. If they were satisfied they will return, be long term loyal customers (and profitable) and recommend to others. If they were dissatisfied they may still have purchased, but they likely won’t return and they may even tell others about the bad experience.
Who came out on top?

• Netflix                       85
• QVC                          85
• Amazon                     83
• Barns&Noble              82
• DrsFosterSmith          81

The Details. Click here for the full free report.