FREEDyourMind

How the Customer Experience Helps .Gov Reform Efforts

by Dave Lewan, Vice President

In April, 2011, President Obama issued General Order 13571, “Streamlining Service Delivery and Improving Customer Service,” part of the Campaign to Cut Waste. It requires federal agencies to save taxpayers money and time by eliminating or consolidating redundant websites and improving outdated, difficult to use, or poorly maintained sites.

Based on lessons learned from actions outlined in Executive Order 13571, each federal agency was directed to develop a comprehensive Web Improvement Plan to help streamline their web operations and improve customer service. 

Howard Merkel, who works with me here at ForeSee on the federal government team and Marianne Klinker, the social media specialist here at ForeSee, noticed how many of the Web Improvement plans mention ForeSee and the work that we do with federal agencies.  It is pretty cool to see them all in one place, so I wanted to share some of them.  

• “Improving the web experience for users is a top priority of the web strategy project that is underway. To establish a baseline, we have implemented the Foresee survey . . . both internally and externally, to assess how well our websites meet user expectations and needs. We use those results to guide our improvements to the customer experience. –Department of Transportation Web Improvement Plan
• “GSA.gov, USA.gov and GobiernoUSA.gov customers are surveyed on the websites using the ACSI survey to benchmark satisfaction on our sites against other government and commercial sites.” – General Services Administration Web Improvement Plan
• “SBA utilizes the ACSI Customer Satisfaction Survey and regularly analyzes the results and areas for further analysis to enhance the online experience across key elements, including search, functionality, navigation, look-and-feel, content, etc.”  – Small Business Administration Web Improvement Plan
• “Commerce operating units use a variety of tools to measure customer satisfaction  . . . We also employ tools that follow the American Customer Satisfaction Index methodology for surveys such as ForeSee.” – Department of Commerce Web Improvement Plan
• “The NRC uses a web measurement survey to enable site visitors to share feedback regarding their experience with our site, in a manner consistent with the American Customer Satisfaction Index. This standardized approach allows the NRC's site performance to be objectively benchmarked against that of other Federal agencies and private industry. We use this feedback to assess and inform our continuous incremental improvements.” – Nuclear Regulatory Commission Web Improvement Plan
• SSA uses the American Customer Satisfaction Index (ACSI) surveys to measure customer/users satisfaction, and to predict future behaviors and trends. ACSI uses a proven, scientific methodology to measure online satisfaction. The data is analyzed and used to help drive decisions on how to improve SSA’s online services.” – Social Security Administration Web Improvement Plan

Citizen satisfaction should be one of several critical components that agencies use when deciding how to streamline content and make federal websites more efficient. And, clearly, it is one of those components, judging by how often it is mentioned as a critical piece of the puzzle by the agencies themselves. Yes, they should also use traffic counts and other metrics to guide decision making, but knowing which sites are highly rated by citizens as useful, easy, transparent is a definite step in the right direction.

I'm really proud of the way our data and analysis is helping in these .gov Reform Efforts. I believe in this stuff, I really do. I wouldn't be here if I didn't. ForeSee’s research has shown that citizens who are highly satisfied with a federal government website rate their trust in the agency 58% higher and are 47% more likely to participate in government than citizens who are less satisfied. Satisfaction also increases the likelihood that the citizen will recommend the site to others, return to it, or use it as a primary resource as opposed to utilizing more costly channels.

These are all things that make the government more efficient, and efficiency saves us taxpayers time and money. It's great to play a part in that.

Social Media Best Practices for Government

The federal government doesn’t have the same flexibility with implementing social media plans as the private sector. The feds have to deal with archaic public comment laws, bureaucratic approval processes for messaging, and smaller budgets.

However, despite all that red tape, our researchshows that all 15 cabinet-level departments have some presence on the 3 big social networking sites (Facebook, Twitter, and YouTube).

We've been releasing the ACSI E-Government Satisfaction Index since 2003. We're trying to shake things up a bit, and while we still plan to release quarterly updates on citizen satisfaction and citizen ratings of transparency, we're trying to supplement that data with other topics of interest to those who are working in e-government, whether at the federal, state, or local level.

So this quarter, two of our usability auditors (thank you Kamaria Campbell and Kathy Totz) did an exhaustive study of what the 15 cabinet-level departments are doing in terms of social media in order to make suggestions and recommendations to other government departments, agencies, and programs. You can download the full report for free on our website.

In addition to the social media best practices, the report also includes satisfaction scores for 100 federal government agencies and our quarterly Transparency Index, which provides a quantitative measurement of how transparent an agency is perceived by citizens. Rhonda Berg does a great job with the satisfaction and transparency data every quarter.

Here are some highlights of the report:

• All 15 executive departments are participating in the three most popular social platforms (Facebook, Twitter, and YouTube).
• Almost half of the 15 departments have more than one presence on a single social media platform.
• Organizations that have a number of presences on one or more social platforms should create an aggregate social media page that lists all of the options with which citizens can interact with them via social media.
• Citizen satisfaction, at 75 on the study’s 100-point scale or higher since late 2009 (with only one exception in the second quarter of 2010, when satisfaction fell briefly to 74.7), shows no sign of waning and maintains an aggregate score of 75.5 in the third quarter of 2011. 
• The aggregate Q3 2011 online transparency score is 77.3, which has increased from last quarter and is the highest aggregate transparency score measured so far.

What's missing from our list of best practices for government social media activities? Any thoughts? We'd love to hear them. I'd also like to hear it if you have ideas for other topics we can include in future ACSI E-Gov Satisfaction Index reports.  

Recommendations for the .Gov Task Force

This spring, President Obama signed Executive Order 13571, “Streamlining Service Delivery and Improving Customer Service,”  which requires federal agencies to strengthen customer service, specifically the manner in which they deliver services and information via the web.

As part of the president's Campaign to Cut Waste, the effort is designed to eliminate or consolidate redundant websites and improve websites that are outdated, difficult to use, or poorly maintained. The desired outcome? Taxpayer money will be saved AND citizens will have an easier access to the content they need.

The .gov Task Force, which was formed in July 2011, now has the difficult task of paring down at least 25% of the more than 24,000 federal government websites in existence today. While the task force is planning to seek qualitative feedback from the public on which sites to cut, it has yet to establish criteria on which it will base its final decision. It is important for the task force to base its decision on the right mix of quantified data (both behavioral and attitudinal) in order to make uniform decisions across the board.

As the government begins the arduous and important decisions of deciding which websites are eliminated and which are consolidated, three critical components must be in play:

• The volume of traffic on the website and other critical behavioral metrics
• Information redundancy
• The website's value to the citizens who regularly use it.

Web traffic alone isn’t enough to justify to cutting a website because that site may contain valuable, highly targeted content that isn’t available anywhere else. Considering whether the site duplicates content is another important consideration, but determining overlaps will be long and arduous with 24,000 sites, or even just with the roughly 1700 federal domains. Therefore, an assessment of the quality and value of the specific sites to citizens is a critical component to determining whether a site is necessary.

Right now, more than 230 federal websites work with us to measure, monitor, and improve citizen satisfaction, and the ACSI methodology that we use to power our technology has been the standard metric for e-gov sites since 2003. We've collected more than 7 million surveys from visitors to federal websites since 2001, including more than 4 million since President Obama’s 2009 inauguration. There is data available on many sites that could guide these kinds of decisions, and certainly any or all of the 1700 .gov domains or the 24,000 federal websites in existence today could be measuring the citizen experience as one mark of success.

We released the second quarter ACSI E-Government Satisfaction and Transparency Index yesterday, with satisfaction scores for more than 100 federal websites and transparency scores for more than 30 websites. You can get it on our website for free and see that there is a compelling case to be made that not only can federal websites be an efficient, cost-effective means of the government meeting its citizenry's needs, but that the means to evaluate, consolidate and improve them are already available. We're also hosting a free webinar to look into the ACSI E-Gov Index Q2 results in more depth... we'd love to have you join us and ask any questions that you may have.

In the meantime, are there other criteria you think the .gov task force should be using to consolidate and pare down websites? Leave your thoughts in the comments.

Transparency 2010: A year in review

We at ForeSee Results recently released our latest e-gov report, focused on transparency and open government.  Transparency and Open Government has been one of the buzz-phrases of President Barack Obama’s White House since the new administration took office two years ago.  This effort was a very ambitious and sweeping objective to transform government departments and agencies into entities that reflect the democratic ideals of openness, democratic participation, and collaboration.  Two years after the initial memo published by President Obama and his administration many are wondering if the goal has taken root and, if so, how successful it has been. In the meantime, political power has changed hands, we’re in the middle of a contentious budget season, and the focus both now and in the future on transparency as a government-wide priority is uncertain.

This is the fifth quarter in a row that we have published such a report and this report gives us the opportunity to look back over 2010 and assess a full year of transparency. In the first quarter of 2010, 23 federal sites participated in the E-Government Transparency Index. By the end of 2010, participation had increased 35%, bringing the total number of participating websites to 32.

Online Transparency scores have been improving for the past two quarters, after a drop in the second quarter of  2010. 

After some extensive research, ForeSee Results developed the E-Government Transparency Index in 2009. The goal was twofold: 

• To create an accurate, actionable and precise measurement of citizens’ opinions on government transparency 
• To quantify the relationships among online transparency, citizen satisfaction, trust, and the likelihood to participate and collaborate with government agencies

The ForeSee Results E-Government Transparency Index has grown into a vital tool for measuring the success, failure, or progress of government departments and agencies online, providing a clear direction for improvement, and forming a benchmark from which all other non-participating entities can draw instruction and inspiration. After all, you cannot manage what you do not measure.

Using a standard methodology that has been in use in government since 1999 (the American Customer Satisfaction Index), the E-Government Transparency Index measures online transparency according to citizen perceptions. How easy is the information to find? How quickly is it made available? How thorough is it? In a democracy, citizen perceptions define reality, and transparency’s success or failure should be first and foremost defined by citizen opinions.

 

Some groups have done exceptionally well.  We use a score of 80 (on a scale of o to 100) as a threshold for a top performer.  The top performers are listed below.

 

There have been others in and around government that attempt to define transparency based on the amount of data that is put in the public domain.  I disagree.  The best measurement of transparency has to be from the citizen's perception. The information needs to be timely, thorough and available to citizens.  Ultimately they are the judge as to whether or not government is transparent.

You can get the full report here.

 

 

Citizen Satisfaction with e-Gov Stays Strong

Strong Satisfaction Scores Continue

Across over 100 Federal Government websites and over 330,000 visitor surveys, satisfaction with Federal Government websites registered a score of 75.1 on a scale of 0-100. While this is slightly off last quarters score of 75.2 (an all-time high), this is not a significant difference.  The fact that citizen satisfaction stayed strong is very good news.

Why does it matter if citizens are satisfied with Federal Government websites?  Is there competition?  If you have a bad experience with the IRS.gov are you going to pay your taxes in Canada?  No.  The traditional way we look at competition doesn't apply to Federal Government websites.  Yet, citizen satisfaction is very important for many reasons.

Channel Competition

One of the main reasons satisfaction with Federal Government websites is important is that the web is one of many channels that citizens can use to get information from the government, perform transactions with the government and communicate with the government.  It is also one of the most efficient ways to accomplish those tasks.  To service the next 10,000 citizens who need information on the Swine Flu from CDC, want to apply for Social Security benefits, get information on Medicare Prescription Drug plans, check out the latest happenings at NASA, investigate our National Parks or do research on a medical condition at Medlineplus, the web provides the most cost effective and efficient way to do so.  Think about trying to accomplish those tasks in a call center or a branch office -- expensive and inefficient.  Citizens will use the channel that provides them the best experience.  So the web is competing with the other channels, we call that Channel Competition.  And the reward for winning the competition?  Lower cost to provide information and services.  A true win-win.  Citizens get more information and services that is more accessible.  And Government is able to provide that information and services at a lower cost.  Which is not only a win for Government but also for us taxpayers.  I guess that makes it a win-win-win.

Highlights

Some highlights from the results include strong scores from many agency websites.  Social Security had two sites that score a 90 on a scale of 0 to 100, the Retirement Estimator (www.ssa.gov/estimator) and Social Security iClaim (www.socialsecruity.gov/applyonline) sites.  These scores are incredible.  To put it in perspective, Amazon and Google scored 86 the last time their ACSI scores were published.  Is Social Security site better than Amazon and Google?  And by better we mean, more functions, better look and feel, more usable, better site performance, etc.  Most experts and everyday citizens would not claim that the Social Security sites are "better" than Amazon and Google, but compared to their expectations they are.  The simple definition of satisfaction is a combination of what you get and what you expect.

In total, 27% of the sites measured scored 80 or higher.  80 is considered the threshold for a top performing site.  Now these sites can not rest on their past success.  As a result of their strong performance and the strong performance of many private sector sites, the standard that visitors will hold these sites to will continue to increase.  And for those sites that were not in the top echelon, they are taking the all important first step to improvement, which is measuring where you are today.  You cannot manage what you do not measure.  

The Satisfied Citizen

We define the satisfied citizen as those citizens who had a satisfaction score of 80 or higher for the e-Gov site they were visiting.  The dissatisfied citizen had a satisfaction of 69 or below.  The impact of satisfying citizens is huge.  The satisfied citizen is 80% more likely to use the the site as a primary resource, 79% more likely to recommend the site to others and 52% more likely to return to the site when compared to a dissatisfied citizen.  The satisfied citizen is also 50% more likely to participate in government and 54% more likely to trust government.

The Full Report

You can find all the details of the report by downloading our commentary on the results or email me and I will send you a copy. 

Social Security Website Better Than Amazon, Google and Netflix!

Wow, what a headline.  And it's true.  At least from user's perspective.  We released the Q4 American Customer Satisfaction Index (ACSI) E-Government Satisfaction Index this week and the results were very good. 

Across over 100 Federal Government websites and over 250,000 visitors surveys, satisfaction with Federal Government websites registered a score of 75.2 on a scale of 0-100.  Equaling the highest score ever registered by the ACSI E-Gov Index last quarter.

But more impressive then that was 27% of the sites measured scored 80 or higher with the Social Security Retirement Estimator (www.ssa.gov/estimator) and Social Security iClaim online benefits application(www.socialsecurity.gov/applyonline) both scored 90.  To put that in perspective, in the last published ACSI results covering the e-business and e-commerce sectors included Amazon at 86, Google at 86 and Netflix at 85.

You can download the full report here or email me and I will send you a copy.

So does that mean the Social Security site is better than Amazon's, easier to use than Google's, has more functionality then Netflix?  Not necessarily.  But more importantly it means that visitors to the site are having a better experience and are more satisfied with the experience on the Social Security site.  The site is doing a better job compared to their expectations.

Now, I hear other self proclaimed experts go on and on about what sites are the best based on their personal opinion.  Not to say their opinion is worthless, but it is worth no more than any other site visitor.  In fact, most "experts" are not "normal" visitors.  Listen to your site visitors, measure their satisfaction and know their expectations.

Congratulations to the folks at the Social Security Administration and the rest of the sites that scored 80 and above, the threshold for a great performing site.

Are Healthcare Websites Sick?

As the debate on healthcare continues, we decided to look at the healthcare websites that we measure at ForeSee Results and see how they are meeting the needs of consumers.  You can download the full report here.

Why look at consumer satisfaction of healthcare sites?  Well, as the debate goes on, one thing is certain, healthcare costs a lot of money.  And one way to reduce and manage some of those costs is better use of technology, for example, improving the quality and effectiveness of the healthcare websites.  The web brings us the opportunity of a win-win that is rarely available in other channels.  The first win is providing more information to citizens when they need it, anytime from anywhere you can get to the internet.  More information that is more readily available to consumers is a big win for consumers.  The second win is cost.  By providing more information, be it educational information about symptoms of a disease, potential side effects of a drug, information about the services provide by a hospital or information about deductibles and co-pays of your health insurance, we can lower the cost of providing the information to consumers.  A more informed consumer will make better health care decisions and lower the total cost of the servicing of that consumer.  A win - win!

So, how did the healthcare sites we measured do.  First of all, we surveyed more then  40,000 consumers who visited these healthcare sites in October and we utilized the proven methodology of the American Customer Satisfaction Index (ACSI) that was founded at The University of Michigan to measure the consumers satisfaction with the websites and how it would impact the consumer's future behavior.  We segmented the sites into four categories and their satisfaction scores are as follows:

• Government Health Information Sites     79
• Pharmaceutical Sites                           78
• Hospital Sites                                      73
• Health Insurance Sites                          64

The aggregate across all sites was 75.  To put these numbers in comparison, we look at 80 as the threshold for a very good site.  On average, the over 700 sites measured everyday by ForeSee Results, resulting in over 1 million completed surveys per month, score a 72, 4% below the healthcare index.  Not bad! 

The sites measured were wide ranging in their success to date, with scores as low as 42 and as high as 86.  86 is the same score that web leaders Amazon and Google obtained in their most recent published ACSI report.  Very impressive results from some of these healthcare sites.

I applaud all the sites (even the ones scoring in the 40's) because the first step to improvement is to measure.  You cannot manage what you do not measure.  And if you don't measure it you won't know if you improved.  So those low scoring sites are at the beginning of a journey that is bound to take them nowhere but up.

So, does satisfaction really matter for these sites?  Well, we talked about the win-win earlier, but lets take another perspective on it.  When we looked across those 40,000 consumers surveyed in October on these healthcare sites, we see that there is huge payback for high satisfaction.  When we look at the consumers who scored 80 or higher and compare them to those that scored 69 or lower, across all the sites, we see that the highly satisfied consumer is:

• 139% more likely to return to the website
• 169% more likely to recommend the website to others
• 153% more likely to use the website as a primary resource for interacting with the healthcare organization

This is truly a huge payback for satisfying your customers and results in the win-win, more information and services at a lower cost.

Satisfaction with E-Gov Hits New High

Great news from the American Customer Satisfaction Index (ACSI) E-Government Index today: an all-time high. In fact, the highest score in six years of measurement and the biggest quarter-over-quarter increase ever.

Download the full, free report here.

So, to what do I attribute the increase?

1) We are almost a year into the most tech-savvy presidency in history. At times, it has seemed to me that President Obama has struggled to find footing with social media recommendations that could confuse priorities and shift attention away from the core business of e-government: communication information via the web. But I think this study is pretty convincing evidence that as he settles into his stride, his leadership is improving e-gov. The score didn't go up just a little bit. It went up 2% and almost 2 points to 75.2. This represents almost 300,000 citizen surveys, so an increase of two points and 2% is significant.

2) We did some analysis and found the longer a federal website has been measuring satisfaction, the bigger the improvements, over both the long term and short term. Those that have been measuring 5 or more years had the biggest increases, followed by those who have been measuring for four years, and so on. After all, you can't manage what you can't measure. So while the Obama administration deserves some of the credit, the truth is that the sites with the biggest increases have been watching these issues for YEARS, long before President Obama was even elected to national office. Federal web managers have been working HARD on this stuff, and there is a core group running e-gov that is really committed to listening to Voice of Citizen. They are committed to using e-gov to better serve the citizenry, and they can't do that without understanding what citizens want and need from their government online.

It's an exciting time for e-gov. While we will doubtless see ups and downs over the next quarters, satisfaction overall has been on the upswing, even outperforming some private sector industries. E-gov is one of the things that the federal government seems to be doing really well for the most part. There is always room for improvement, especially if expectations continue to increase. But there are a lot of individual sites doing very, very well.

Latest E-Gov Satisfaction Scores out on Tuesday.

I can't wait to share the latest American Customer Satisfaction Index (ACSI) scores for e-government due out Tuesday.  The results are remarkable.  We are now three quarters into the new "technology savvy" administration, and it will be interesting to see if this has had any impact on the satisfaction scores. 

This report is useful and interesting for a couple of reasons:

1) We can see the trend in citizen satisfaction with e-government overall. The trend over the last six years is generally up, though the recent trend has been stagnation. Tuesday's report will shed light on whether any of the efforts of the Obama administration have come to fruition in terms of improving citizen satisfaction with e-gov initiatives.

2) We can see how indivdual websites are doing compared to each other, compared to the private sector, and compared to themselves over time. There are 104 individual government websites included in the report from virtually every agency and department. Each website gets a score on a 100-point scale; the higher the better. Anything over 80 is generally considered excellent. Which websites have made the biggest improvements over the last quarter? Over the last year? Which sites are better than Amazon and Google? Which sites aren't doing so well?

3) We can quantify the role of online satisfaction on citizens' likelihood to return to the site, recommend it, even use it as a primary resource. We can do this in aggregate for the federal government overall, and for individual sites.

The data is being compiled and reported as we speak--everyone is busily getting everything together for the release on Tuesday.

This report is useful and interesting for a couple of reasons:

1) We can see the trend in citizen satisfaction with e-government overall. The trend over the last six years is generally up, though the recent trend has been stagnation. Tuesday's report will shed light on whether any of the efforts of the Obama administration have come to fruition in terms of improving citizen satisfaction with e-gov initiatives.

2) We can see how indivdual websites are doing compared to each other, compared to the private sector, and compared to themselves over time. There are 104 individual government websites included in the report from virtually every agency and department. Each website gets a score on a 100-point scale; the higher the better. Anything over 80 is generally considered excellent. Which websites have made the biggest improvements over the last quarter? Over the last year? Which sites are better than Amazon and Google? Which sites aren't doing so well?

3) We can quantify the role of online satisfaction on citizens' likelihood to return to the site, recommend it, even use it as a primary resource. We can do this in aggregate for the federal government overall, and for individual sites.

The data is being compiled and reported as we speak--everyone is busily getting everything together for the release on Tuesday.

The Role of the Web in Government - what do 81 million people want?

Did you know that 81 million Americans visited a Government Web Site in July?  This is according to a comScore release in early September.  If I am not mistaken that is more people then voted for our current President.

So what does this mean? There is a lot of buzz and activity going on e-gov these days, from cloud computing to social media to the talk of transparency.  But 81 million people visiting government websites in the month of July alone.  WOW.  That is a lot of traffic.  About 1/2 of the traffic the number of visitors to Google sites in July and almost the same amount of visitors to Facebook in July. 

Government agencies need to keep great focus on their websites.  There is where they can have the most influence on citizens.