FREEDyourMind

The "World" in World Wide Web

Ok, we all know what the world wide web is. But what about the world coming to our doorstep, and the world coming to our neighborhood?

According to recent research from Pew Internet & American Life Project, Latinos comprise 14% of the U.S. adult population and about half of this growing group(56%) goes online.

Are we ignoring this fast growing group? Does your site have a Spanish alternative? And if your site does have a Spanish alternative, are you surveying those customers to make sure you’re meeting their needs just like you survey your English-speaking customers?

1.      Don’t assume you can just translate your content directly from English into Spanish and be done with it. Don’t assume they want or need the same exact information. Our client GobiernoUSA.gov (the Spanish version of USA.gov) found that their Spanish-speaking constituents needed information on immigration and VISAs that visitors to USA.gov didn’t need as prominently displayed. In fact, they found they had an entirely different audience than their English counterpart, USA.gov.

2.      Don’t assume the same things will satisfy or dissatisfy them. You need to ask all your customer segments in their own language what they need and want from your site; then you need to develop content accordingly. Your English speakers may find the navigation is cumbersome but the search function is fine, while your multilingual visitors may have the opposite experience.

3.      Don’t survey multilingual visitors in English. You’d be amazed at how many people make this basic mistake. It reverses any goodwill you get from asking their opinion in the first place if you pop up an English survey on a Spanish site.

Leilani Martinez at GobiernoUSA.gov likes to make the point that creating an excellent multilingual site is not rocket science. If you know how to create a website, you know how to do it in Spanish. People are too easily intimidated by an audience they are less familiar with. All the more reason to survey all your customers directly and let them guide you!

Response to Avinash

Avinash Kaushik wrote a great post this week on how you should choose an online survey provider.

Avinash worked extensively with ForeSee Results, so he’s very familiar with the ACSI methodology and how we apply it online. Since he’s been both a customer of web metrics and a partner of web metrics companies, he has a lot of knowledge and has given a lot of thought to what kinds of things people should consider when choosing a tool.

I just want to expand on a few of Avinash’s points:

#1] Mathematical Rigor: No matter what you choose look for a partner that can apply mathematical rigor to your results. I find so many results mis-interpreted because poor math applied in the analysis. Measuring survey results is not simply taking the average of the answers (averages lie!), it is measuring distributions and doing regressions. You don’t want to be bothered to apply the statistics and statistical significance, stress test to ensure your vendor does (then you can focus on analysis and not reporting).

This is a great point that too many people undervalue (just look at the Net Promoter fad: margins of error in the double digits, poorly-designed distributions, and people love the “simplicity” of it!). But I would take this point about mathematic rigor even further. It’s not enough JUST to have mathematical rigor: you can ask bad questions and have a survey pool that is not representative of your audience and you can still use mathematical rigor to interpret the results and get a low margin of error and high confidence intervals. Mathematical rigor ALONE is not what leads you to the holy grail; what leads you there is having a methodology behind the survey that is accurate, precise, and predictive and THEN making sure you analyze those results with mathematical rigor.

This brings me to a comment I had on Avinash’s point #3 (skipping past #2, which I’ll come back to because #3 is related to what I’m already talking about):

#3] Benchmarks & Indexes: Few people in senior management will take action when you tell them “our score on satisfaction (or content or navigation) is 6.0″ (or 45, depending on your vendor). But most of them will get off their butt and give your money to take action when you go to them and say “our score on satisfaction is 6.0 and amazon.com is 9.4 out of 10″.

 

Your actual score does not drive action, the difference between that and a industry benchmark drives action (no one wants to look bad by comparison!). And here is another subtle human factor: no one wants your opinion about anything, providing a comparison to a external benchmark depersonalizes the number and it is more likely that it will be “heard”.

 

 

iPerceptions uses the iPSI (developed by iPerceptions), ForeSee user the ACSI (developed by University of Michigan), both wonderful benchmarks you can compare your scores to and help motivate your management to take action! There are other vendors, look to see if your vendor will provide you with a benchmark or a index to compare.

 

Another great point, especially about the executive tendency to listen up when you can create an aura of rivalry with a competitor on industry leader. But again, just having a benchmark isn’t enough—anybody can create a methodology and an index and a benchmark. I could create the Larry Satisfaction Index—LSI ™ , write a white paper on it, and call it a methodology, but that doesn’t make it MEAN anything unless it is accurate and precise. An analogy: you’re on the golf course, you look down the fairway and decide it’s 120 yards to the green. You take out your nine iron and hit a perfect shot: exactly 120 yards. But you’re still 30 yards short of the green. You weren’t 120 yards from the green, you were 150 yards! You had a measurement, you had a methodology, but it was worthless because it wasn’t accurate, reliable, or precise.

 

To to this point about benchmarking, I would only add that people should make sure that whatever benchmark they chose has a proven history of being credible, accurate, precise, predictive, and reliable. What is a proven history? Well, the proof is in the pudding—there should be scientific, academic, published proof that the metrics are linked to financial performance and desirable future behaviors like word of mouth and loyalty. Just ask your vendor to show you the proof that their benchmark actually means something. The American Customer Satisfaction Index methodology has that proof.

 

Back to Avinash’s #2, about on-the-fly segmentation capabilities:

 

#2] On The Fly Segmentation Capabilities: Look for the capabilities that are provided to do on the fly segmentation of your data (aggregates lie!) . . .

There is gold in your ability to pick a particular segment of traffic that absolutely hates you and slice it off and drill down to why they were on your site, what products they own, what did they not find, and… then go fix it fast. Segmentation rocks! You want to be able to do it efficiently, yourself if possible, to reduce chances of vendor delays.


My one addition to this is that some companies may want to choose a company who will let you do the segmentation yourself, but it may be that what you really want and need is a seasoned team of experts to do this for you or help you do. Data overload is a big problem with metrics, and sometimes companies plan to do it themselves and end up doing nothing. So look for a vendor that can enable you to do that segmentation OR that can do it for you.

Last, but not least, Avinash mentions page-level vs. site-level surveys in his discussion of pre-survey considerations:

 

Page level surveys are user initiated (”tell me what you think of this page” “rate this page” etc) and serve the purpose of measuring the experience of the page. You can ask “site” level experience questions but the way they are initiated by users and local level at which initiation happens makes them a sub optimal choice for measuring site experience. They can good for measuring effectiveness of a page.

 

Site level surveys are usually presented to the users (”please give us a few minutes to answer these questions”), typically on exit, and are a great measure of the site experience (”what got you here today” “what were you able to do” “how much did we suck” “did you find what you were looking for”). They are good at measuring effectiveness of a site.

I would disagree slightly: page level surveys are more feedback and less measurement per se, for two reasons:

 

1) Since they are opt-in, they aren’t representative of your larger audience, which means they aren’t a true measurement.

 

2) The page that people give you feedback on tends to be several pages after the page that caused them the problem, because people don’t get immediately frustrated and opt-in to a survey to tell you about it. Frustration builds a little more slowly. So feedback that shows up on page surveys is not necessarily representative of the experience or effectiveness of THAT page in particular.

That’s it. Those would be my additions or clarifications to a very thoughtful post on how to choose a survey vendor. Any of your own to add?

Yahoo Edges Out Google in Customer Sat

Today’s American Customer Satisfaction Index (ACSI) results are all over the news for one big reason: Yahoo beat Google in customer satisfaction for the first time since both companies were added to the Index.

Actually, Yahoo’s (79) one-point lead over Google (78) on the ACSI’s 100-point scale isn’t the biggest story here–the bigger deal is that they’re moving in opposite directions. In 2006, Google had an ACSI score of 81 and Yahoo! a score of 76. This year, Google fell three points to 78 and Yahoo gained three points to 79. I attribute this mainly to the fact that to the average user, Google hasn’t changed all that much in the last few years. Sure, they’ve added incredible new functionality and tools like maps, iGoogle, gmail, calendars, etc., but I don’t think the average user who goes to google.com to do a search knows about those new capabilities. And Google doesn’t make them look very obvious. Yahoo, on the other hand, is making changes that are more useful and accessible to the every day user. Yahoo’s increase should be good news for CEO Jerry Yang, since ACSI scores are a proven predictor of financial performance, stock prices, loyalty, and recommend.

The other big news is Ask.com’s huge increase: up 5.6% from 71 to 75. They relaunched without paying the typical price (usually satisfaction scores drop in what we call the “relaunch effect”), which shows they did so with the right mix of evolution and revolution. Their market share is still small, but their satisfaction score has increased 21% and 13 points since they were first measured as AskJeeves.com in 2002, which is a phenomenal gain in such a short time.

AOL fell 9.5% from a 74 to a 67 in this year’s ACSI report. MSN increased marginally from 74 to 75.

The report also measured the top news and information sites, like ABCNews.com (74), MSNBC.com (74), CNN.com (73), NYTimes.com (73) and USAToday.com (72).
Get the full report here.

St. John's Use of Customer Sat Metrics

Check out the recent article about the value of measuring customer satisfaction online, written for Health Leaders Magazine by Pam Hedman at St. John Health System, a group of 8 hospitals and more than 125 medical facilities in Michigan. St. John is a client of ours–they’ve been measuring customer satisfaction online for about four years now, and wanted to share how customer satisfaction metrics helped them figure out that:

• the composition of site visitors was entirely different than what they expected, requiring changes to the site to address the needs of very specific audiences that weren’t previously considered significant
• improvements      to search more than to any other area of functionality would have the      greatest ROI
• improvements needed to be made to a transactional component of the website, resulting in a 138% increase in site traffic and a much better understanding of what was driving customer acquisition and retention.

Pam also talks about how she used customer satisfaction data to justify budget decisions and secure the funding she needs for web initiatives. Thanks, Pam, for the great article!

Keeping Up With the Joneses

I met with a CEO last week who was only interested in one metric: how his company performed in terms of customer satisfaction when compared to their nearest competitor.

Comparing yourself to your can be an important reality check (and every company should monitor their performance relative to competitors as part of a dashboard of metrics), but it shouldn’t be a guiding principal or sole metric. Let’s not lose the forest for the trees.

Let’s say your competition has a score of 75 for navigation and yours is a 71. Does that mean yours needs to be better? Maybe. But more important is to see if improving that score of 71 in navigation will impact their behavior. Will they buy more, return more frequently or recommend to more people? The impact of changing is what should drive your decision, not just the comparison of the scores. It is about impacting the behavior of your customers and prospects, not keeping up with the Joneses? Is there another area you could improve (product descriptions, search, etc.) that would have a bigger impact on your customers’ likelihood to buy, recommend, and return?

Don’t spend too much time chasing your competitor; spend your time chasing your customers and prospects! Improving their satisfaction will yield significant improvements to your bottom line.. Satisfy your own customers in the ways that are most important to them, and use competitive benchmarks as one piece of the puzzle.

Is Anyone Really Surprised?

The latest mini-corporate scandal-in-the-making is CEO’s who participate anonymously in online message boards, lauding their own companies and disparaging competitors, all while pretending to be someone else with no vested interest in the company’s fortunes. The Whole Foods CEO started the trend (of getting caught, at least).

It’s fun to wag our fingers and chastise leaders who should know better, and we ALL agree this wasn’t smart, especially the bits about trying to influence investment in your own company vs. in a competitor.

But what it really points out is a lack of transparency in word of mouth (WOM) marketing–a tool that more and more companies are relying on and looking at as scientific. At the end of the day, people will take advantage of any open system no matter how much hand-wringing the WOM community does about ethics and best practices. WOM is valuable, and for now it still works. As long as that’s true, there will be people manipulating it for their own means, just like everything else in human history.

Most savvy internet users know that when they’re reading reviews on Amazon.com, a decent proportion of them come from BzzzAgent, a company that pays readers to review certain books that want to create a word of mouth buzz. When you read reviews on iTunes, how many of them came from the marketing department of the band’s record label? And when you read a Yahoo! investment message board lauding one stock over another, did it really come from your average Joe investor (as seems to be the case) or from an anonymous board member, CEO, or shareholding employee pretending to have no unusual interest in the subject at hand?

The point is that you don’t know and you can’t know. Reading opinions on a message board isn’t like chatting with your neighbors at a cookout, where you can see who is saying what and have at least some vague sense of any ulterior motives when they pick or pan some book or restaurant or cell phone or stock.

Internet users need to take all anonymous WOM recommendations they read with a huge grain of salt, and marketers need to prepare for the day when these kinds of tactics won’t work anymore because people are on to the fact that they’re essentially being gamed.

More on Customer Reviews

Walmart.com recently announced that they’re jumping on the bandwagon to offer customer reviews themselves, both on the product page itself and via email after they’ve gotten the product mailed to them. The Walmart.com CMO said that in the future, Walmart.com could interact with people who posted reviews and use the reviews to decide what Wal-Mart sells.

 

One word of caution: understand the real value and role of customer reviews and do not overestimate their abilities. I just did a post about our research into customer reviews, and how we found them to be beneficial to customer loyalty, customer satisfaction and future sales. There’s no doubt that customer reviews are a great thing, and I’m a big proponent of any retailer adding reviews to their site. But the reason they’re good is because they will give customers valuable information, not because they will necessarily give retailers valuable information.

 

 

Customer reviews are a great tool for customers, but they are not necessarily a valuable metric for companies or a great tool for measuring the quality of a product. So 1% of your customers say they love something. Is it the 1% that own stock in your company? Is it the 1% that spends the most money with you? Is it the 1% who works for the manufacturer of the product in question, trying to artificially inflate reviews for the product?

 

Also, customer reviews tend to draw out the extremes—those that either really love or hate something. Walmart.com’s own research showed that in a testing phase, most reviews were positive: 4 or 5 stars (out of a possible 5). That’s not surprising: you tend to see a lot of 1’s and a lot of 5’s but very few 3’s.

 

If you want to measure customer reaction to a product, you need to use a satisfaction-based model and it needs to be random, not opt-in like customer reviews.

Customer reviews are not a product development or a product management tool, they’re a marketing tool. Understand their value to your customers, don’t overestimate their value to you, and they can and should be a great addition to any retail website.