FREEDyourMind

Facebook Flops, Google Drops, FoxNews Mops Up the Floor with CNN

So you may have seen by now that the big #ACSI report on e-business was released today (hyperlink to http://www.foreseeresults.com/research-white-papers/ACSI-e-business-report-2010.shtml) with a bang.

We've been doing the analysis for the ACSI since 2002, and there is usually pretty big news in one of the measured e-business categories, but this year, there's a shakeup in all three.

First thing's first: social media. This is the first year the ACSI has measured social media, and what a debut. With an average score of 70, social media has the lowest industry aggregate score of any of online industries measured by the ACSI. Wikipedia leads the social media industry with a score of 77, and YouTube comes in at 73. Industry darling Facebook registers a score of 64, only one point above MySpace (63). Facebook and MySpace have scores so low, the only other companies measured by the ACSI with such low scores are airlines and cable companies. Facebook seems to be suffering from concerns about privacy, frequent changes to the interface, and advertising.

	2010
Social Media Aggregate	70
Wikipedia.com	77
YouTube.com (Google Inc.)	73
All Others	72
Facebook.com	64
MySpace.com (News Corp.)	63

As for News and Information Sites, the big news in this industry is the debut of FoxNews.com at the top of the heap with a score of 82, which is five points above nearest competitor USAToday.com (77) and well ahead of other cable news providers MSNBC (74) and  CNN.com (73). NYTimes comes in at 76. FoxNews registers the highest score ever received by any news organization in the ACSI.Our data shows the high score could be in part because their viewers are very loyal and don't follow as many different news sources as users to the other measured outlets. Therefore, their expectations may be a little lower, which makes them easier to satisfy. But whatever the reason for their high satisfaction, it is off the charts for a news site.  If I were Fox, I’d be using this data in my media kit, because high satisfaction leads to high return visits and loyalty, which means more eyeballs for ads on FoxNews.com.

The Portals and Search Enginescategory is usually a pretty stable in terms of who dominates the field: Google has been at the top every year but this year and in 2007 (when Yahoo slipped momentarily ahead). This year, the "all others" group beats Google, which is a combination of all the other search engines not mentioned by name in this study. This year, they tended to be niche search engines. Google itself had a 7% decline this year (down six points to 80). Bing was measured for the first time this year, and it made a strong first showing with a score of 77, second only to Google.

So what do you think:

• Can Bing actually compete with Google, even with all the power of Microsoft? Last year I would have said no; this year I think Bing might actually have a glimmer of hope.

• Does Facebook's low satisfaction score pose any danger for the company? Or like other perennially low scoring companies like airlines and cable companies, does it have enough of a specialized market that it can dominate despite low satisfaction?

• Why can't the other cable news channels come close to FoxNews.com's high satisfaction score?

I'd love to hear what you think in the comments...

You can download the free report here: http://www.foreseeresults.com/research-white-papers/ACSI-e-business-report-2010.shtmlor catch me on Twitter (@larryfreed) with any questions about the findings.

Are Healthcare Websites Sick?

As the debate on healthcare continues, we decided to look at the healthcare websites that we measure at ForeSee Results and see how they are meeting the needs of consumers.  You can download the full report here.

Why look at consumer satisfaction of healthcare sites?  Well, as the debate goes on, one thing is certain, healthcare costs a lot of money.  And one way to reduce and manage some of those costs is better use of technology, for example, improving the quality and effectiveness of the healthcare websites.  The web brings us the opportunity of a win-win that is rarely available in other channels.  The first win is providing more information to citizens when they need it, anytime from anywhere you can get to the internet.  More information that is more readily available to consumers is a big win for consumers.  The second win is cost.  By providing more information, be it educational information about symptoms of a disease, potential side effects of a drug, information about the services provide by a hospital or information about deductibles and co-pays of your health insurance, we can lower the cost of providing the information to consumers.  A more informed consumer will make better health care decisions and lower the total cost of the servicing of that consumer.  A win - win!

So, how did the healthcare sites we measured do.  First of all, we surveyed more then  40,000 consumers who visited these healthcare sites in October and we utilized the proven methodology of the American Customer Satisfaction Index (ACSI) that was founded at The University of Michigan to measure the consumers satisfaction with the websites and how it would impact the consumer's future behavior.  We segmented the sites into four categories and their satisfaction scores are as follows:

• Government Health Information Sites     79
• Pharmaceutical Sites                           78
• Hospital Sites                                      73
• Health Insurance Sites                          64

The aggregate across all sites was 75.  To put these numbers in comparison, we look at 80 as the threshold for a very good site.  On average, the over 700 sites measured everyday by ForeSee Results, resulting in over 1 million completed surveys per month, score a 72, 4% below the healthcare index.  Not bad! 

The sites measured were wide ranging in their success to date, with scores as low as 42 and as high as 86.  86 is the same score that web leaders Amazon and Google obtained in their most recent published ACSI report.  Very impressive results from some of these healthcare sites.

I applaud all the sites (even the ones scoring in the 40's) because the first step to improvement is to measure.  You cannot manage what you do not measure.  And if you don't measure it you won't know if you improved.  So those low scoring sites are at the beginning of a journey that is bound to take them nowhere but up.

So, does satisfaction really matter for these sites?  Well, we talked about the win-win earlier, but lets take another perspective on it.  When we looked across those 40,000 consumers surveyed in October on these healthcare sites, we see that there is huge payback for high satisfaction.  When we look at the consumers who scored 80 or higher and compare them to those that scored 69 or lower, across all the sites, we see that the highly satisfied consumer is:

• 139% more likely to return to the website
• 169% more likely to recommend the website to others
• 153% more likely to use the website as a primary resource for interacting with the healthcare organization

This is truly a huge payback for satisfying your customers and results in the win-win, more information and services at a lower cost.

Latest American Customer Satisfaction Index Results Released Today

Today the American Customer Satisfaction Index (ACSI) released its latest results on customer satisfaction.  You can read the full commentary here.   Covered in this quarter's results are some of my favorites - Beer and Chocolate.  The categories covered in the Manufacturing Nondurable Goods Sector include Apparel, Athletics shoes, Breweries, Cigarettes, Food Manufacturing, Personal Care & Cleaning Products, Pet Food and Soft Drinks.

If you follow the American Customer Satisfaction Index or ForeSee Results, you know that Customer Satisfaction as measured by the ACSI at a macro level is a leading indicator for economy (Consumer Spending, GDP Growth, etc.) and at an individual company level is a leading indicator for financial success.  The ACSI was founded at the University of Michigan.

So, what does the future hold?

Overall, the National ACSI score dipped ever so slightly from 76.1 (the all time high) last quarter to 76.0 this quarter, a very minor 0.1%.  And it is up a significant 1.3% from Q3 2008 when it had a score of 75.0.  The Q2 ACSI results predictive model pointed to an increase of consumer spending of 3.25% , much higher then most others were projecting and actual Q3 consumer spending growth turned out to be 3.35%.  With a flattening of the score this quarter it is projecting consumer spending to be between 2% and 3% in Q4.

Some of the companies that really excelled in these results include Levi Strauss rising 6.4% to a score of 83, Anheuser-Busch rising 3.7% to 85, Mars, Hershey and Nestle all seeing increases between 1.2% and 2.4% with scores between 85 and 87.  So, America is going back to basics - "non-designer" Jeans, Anheuser-Busch and Chocolate!  Not a total surprise, when the economy is tough we tend to gravitate back to what is familiar and what gives us simple pleasures.

Latest E-Gov Satisfaction Scores out on Tuesday.

I can't wait to share the latest American Customer Satisfaction Index (ACSI) scores for e-government due out Tuesday.  The results are remarkable.  We are now three quarters into the new "technology savvy" administration, and it will be interesting to see if this has had any impact on the satisfaction scores. 

This report is useful and interesting for a couple of reasons:

1) We can see the trend in citizen satisfaction with e-government overall. The trend over the last six years is generally up, though the recent trend has been stagnation. Tuesday's report will shed light on whether any of the efforts of the Obama administration have come to fruition in terms of improving citizen satisfaction with e-gov initiatives.

2) We can see how indivdual websites are doing compared to each other, compared to the private sector, and compared to themselves over time. There are 104 individual government websites included in the report from virtually every agency and department. Each website gets a score on a 100-point scale; the higher the better. Anything over 80 is generally considered excellent. Which websites have made the biggest improvements over the last quarter? Over the last year? Which sites are better than Amazon and Google? Which sites aren't doing so well?

3) We can quantify the role of online satisfaction on citizens' likelihood to return to the site, recommend it, even use it as a primary resource. We can do this in aggregate for the federal government overall, and for individual sites.

The data is being compiled and reported as we speak--everyone is busily getting everything together for the release on Tuesday.

This report is useful and interesting for a couple of reasons:

1) We can see the trend in citizen satisfaction with e-government overall. The trend over the last six years is generally up, though the recent trend has been stagnation. Tuesday's report will shed light on whether any of the efforts of the Obama administration have come to fruition in terms of improving citizen satisfaction with e-gov initiatives.

2) We can see how indivdual websites are doing compared to each other, compared to the private sector, and compared to themselves over time. There are 104 individual government websites included in the report from virtually every agency and department. Each website gets a score on a 100-point scale; the higher the better. Anything over 80 is generally considered excellent. Which websites have made the biggest improvements over the last quarter? Over the last year? Which sites are better than Amazon and Google? Which sites aren't doing so well?

3) We can quantify the role of online satisfaction on citizens' likelihood to return to the site, recommend it, even use it as a primary resource. We can do this in aggregate for the federal government overall, and for individual sites.

The data is being compiled and reported as we speak--everyone is busily getting everything together for the release on Tuesday.

In a Blind Search Test, Which Do You Choose?

Our annual ACSI rating of search engines and portals is coming up next Tuesday, and in doing some research about the measured companies, I found this article in TechCrunch about the Blind Search Tool.

Here's the theory: Google gets a lot of credit just for being Google. If you take Yahoo or Bing search results and put a Google logo at the top, people will like it better, but when you compare head-to-head functionality, Bing and Yahoo hold their own. Google's market share hovers around 70% of searches, but in a blind test of which search results are actually the ones you're looking for, people choose Google as the best 44% of the time, Bing 33% of the time, and Yahoo 23% of the time.

This kind of distribution would suggest that there is room for more than one search engine. I did the test four times and chose Bing twice and Google twice in the blind test, requiring a tie-breaker, in which I chose Yahoo. Pretty representative of the numbers above, actually.

It is interesting to look at those results and compare them to the market share numbers.

Which do you choose in the blind search test? And is it time for a Facebook quiz: what search engine result are you? (in other words, enough with the Facebook quizzes, already!) 

Announcing a Beautiful, Bouncing . . . New Technology!

ForeSee Results is proud to announce our latest innovation, CS SessionReplay, introduced today at eMetrics in San Jose.

CS SessionReplay is a brand new technology that lets you record and watch actual site visitor sessions in our Online Portal, enabling you to extend the benefits of your ACSI-driven satisfaction analysis of key visitor segments down to the individual respondent level. In other words, let satisfaction data and open-ended comments show you which sessions will be useful and relevant, and then watch what your users are actually doing on your site, including their mouse clicks and movements, page scrolling, form entry, and all interactivity with a website (all while adhering to the strictest privacy and security standards). It even skips over idle time so that you're just watching the good parts, just like TiVo for your website!

Here's a link to watch a video showing you how it works. Caution: it starts playing as soon as you click the link, so turn your volume down if you don't want to blast everyone away! It's about a minute long. CS SessionReplay Video. (Look closely and you may just see Jim Sterne and Eric Peterson!)

The clients that are already using it are finding it's a great way to get usability insights without the cost or artificial environment of a usability lab. The addition of CS SessionReplay to existing customer satisfaction analytics and usability services allows ForeSee Results to provide a single, unified platform which links recordings of visitor sessions, customer satisfaction scores and impacts, comments and feedback about a site, and third-party data--all in one place--which is good news for companies who want to simplify without losing functionality or analytics capabilities.

We bought the technology from Nitobi last year and have been working furiously to integrate it with our platform and get it ready for prime time. It finally is, and we couldn't be prouder. Great job to everyone on our technology team and product team, who did an amazing job creating a valuable solution for companies struggling in a recession.

Perception is Better Then Reality

I had an interesting conversation at an industry event a short while ago that is worth sharing.  To protect the innocent, let's use the name Sam.  Sam was questioning the value of voice of customer analytics.  His premise was "why do I care what they say, I care what they do.  Satisfaction, attitude and perception are soft and fluffy.  Give me hard data that I can rely on, clicks, hits, page views, conversion rates and sales."

My answer -- perception is reality, better yet, perception is better then reality.

Well, what do you think?  Which is correct?

Let's dive a little deeper.  I can understand where Sam is coming from.  Real numbers are good.  I know when I am looking at our financials I want real numbers, not opinions and perceptions.  I know when I am looking at the shrinking stock market, I want real numbers, not excuses and opinions.  So why do I want perceptions, intent, attitudes and opinions when talking about something as important as the success of my website. 

There is more to the visitor then today.  There is tomorrow - what the visitor will do in the future - will they return, recommend my site to others, will they be loyal?  There has been a lot of work done in years past on the lifetime value of a customer.  That is an important concept. 

What the visitor will do in the future is directly dependent on their perception of the experience today.

The visitor may have bought today.  But will they buy tomorrow?  If they were satisfied they will. If they were not satisfied they won't buy from you in the future - they will go elsewhere.  The visitor's perception of the experience (their satisfaction) will determine what they do in the future.  Their perception of the experience is far more important then the actual and factual experience.

Shopper Satisfaction Holding Steady During the Holidays

We released our second weekly holiday benchmark this morning.

Here's the bad news: satisfaction is still down year over year. But there is a real silver lining, and that is that satisfaction is maintaining from Cyber Monday--it has not fallen off in the week after one of the biggest online shopping days of the year. In fact, purchase intent is higher this week than it was at any point during November. If you're looking for good news in a bleak landscape, this is it!

I think all of us watched the good results from Cyber Monday and wondered if that was it. Did people do all their shopping the weekend after Thanksgiving to take advantage of deals? Would we see shopping fall off the rest of December?

I think we have all adjusted expectations--we are not going to see 15%-20% year-over-year increases like we have in years past. But everything I'm seeing tells me it also won't be as bad as some predicted at the beginning of the season.

But perhaps the bigger story than what will happen to e-retail overall is what will happen to individual retailers. Our benchmark covers more than 80 retailers, and we've collected more than 330,000 e-retail surveys over the last 5 weeks (more than 70,000 just in the week after Cyber Monday). When we're looking at that many shoppers of that many online retailers, there is going to be a huge range of satisfaction--some retailers as high as 90 (on our 100-point scale) and some are, well, a LOT lower than that.

I've said it before and I'll say it again: an economic crisis can be a huge opportunity for e-retailers to solidify loyalty and to really differentiate themselves in terms of customer satisfaction. Really, for anyone in any industry to do so. Satisfied customers are the last to leave, no matter how tight their personal finances get.

KPI's Are Old School: Let's Focus on KSI's

A lot of smart, good, well-respected people talk about key performance indicators (KPI's), including Eric Peterson's excellent Big Book of KPI's. So I don't want to knock KPI's. And for a long time, KPI's have often been an excellent measure, but of performance, not necessarily success.

Maybe this is an issue of semantics, but I don't think so. I want to make a case that KPI's should be supplemented with KSI's (key SUCCESS indicators).

When you're measuring performance, you're looking at what has already happened. Things like how long it takes a page to load, how many visitors, bounce rates, search results . . . all those things reflect what has already happened.

The problem is, we measure what we know how to measure: performance. Measuring success is a much harder thing to get your mind wrapped around. Sure, you can measure sales as an indicator of success, but even that reflects what has already happened. Conversion rate is another good example that tends more towards being an indicator of success than just performance, but again is backward looking. And for that matter, conversion rate can often be a very misleading metric.  If you've ever heard me speak, you've probably heard me ask: how can you steer a car in the right direction if you're always looking in the rear-view mirror?

The bottom line is that behavioral data is backward looking. Important data, yes. Valuable data, yes. Insightful data, yes. But it doesn't help you as much as it could. The other issue is that behavioral data doesn't really measure the success of the experience and therefore the long-term value to the organization.

Satisfaction data, when measured correctly, is forward looking. Academic studies show that certain satisfaction methodologies can actually predict the future. They can tell you what your customers are likely to do tomorrow.

Some examples of KSI's: satisfaction, purchase intent, likelihood to recommend, likelihood to use the website as a primary channel, etc. In other words, KSI's = future behaviors.

The Limitations of Traditional Online Surveys

I could probably write a book with this title (hey...there's an idea!), but the reason I'm thinking of it now is that I was just popped a survey on a magazine website.

It asked all the basic demographic information that might be useful for research. But so much more was left on the table.  It is great to gather information, but if you are going to put your site visitors through a survey, make sure you are making the most of the opportunity. 

Some quick thoughts (or you could call them "must haves"):

• Make sure you are getting a representative audience to complete the survey.  Without it you have individuals opinions (there is some value there) but with a representative audience you have taken the first step towards having a useful metric.
• Demographics can be important,  but lets also understand what is driving their satisfaction and what is likely to result from the online experience (loyalty, retention, purchase online, purchase offline, etc.).
• Understand the relationship between those drivers of satisfaction, overall satisfaction, future behaviors and financial success. It is great to know where we are today, but the holy grail is to know what we need to do to improve -- and what impact it will have on our success.  This is not an easy task, but it is the key to turning voice of customer into actionable information.
• Rely on a proven methodology.

I like to think of it as "applying science to voice of customer."