Today we released our inaugural E-Government Transparency Index. You can
download the report for free on our website and read about it on NextGov. I'll share the
results first and then explain a bit of the background. These are all
Transparency scores, calculated on a 100-point scale. We're starting with 14
federal government websites, so this is far from representative of the whole
government, and there are certainly sites out there with slightly higher and
much lower transparency. But this study does represent more than 30,000 citizen
surveys, which is nothing to sneeze at.
In the Next Gov article, GSA's Dave McClure said, "We have always assumed that greater transparency [and] more openness in government would link to greater satisfaction and higher trust in government. What this study does is help confirm that."
Lots more details in the full report, which you can download for free on our website.
Now, a little more background on what we're measuring and why.
When President Obama released his memo last February about transparency and open government, we immediately got to work. We were already measuring online customer satisfaction with more than 200 federal websites, and our researchers thought there might be a way to adapt our methodology to measure the President's transparency goals in an accurate, precise, actionable manner. If we could do it, it would give agencies a way to measure their performance on these objectives. After all, as I have been known to say maybe once or twice in my career (or in the last 5 minutes), you can't manage what you don't measure. Agencies cannot manage, maintain, or improve transparency, communication, collaboration, or trust without a quantifiable measure of these objectives.
For months, we worked on models and tested them. We were already measuring dozens of drivers, or elements of satisfaction (such as navigation, search, content, look and feel, etc), and we had a theory that online transparency was another one. Citizens who perceive a website as more transparent are going to be more satisfied. Satisfaction as measured by the American Customer Satisfaction Index (ACSI) has been sown again and again to predict certain future behaviors such as likelihood to use the website as a primary channel instead of other, more costly channels; likelihood to recommend, likelihood to return to the website, etc. So we wanted to see if it was also true that satisfaction predicted likelihood to communicate and collaborate with a given agency, and we wanted to see if satisfaction predicted trust in the agency. These were all the key elements laid out in the Obama memo.
After three months of testing the models, we implemented the new methodology on e-gov sites that volunteered to be part of our research, and then we tested some more. We tweaked, we adjusted, and in July, we published our first report, showing conclusively that transparency was a key driver of satisfaction (in fact, for most agencies but not all, it was the most important driver of satisfaction) and that satisfaction, in turn, predicted trust).
Meanwhile, the buzz about our research grew, and agencies were signing up left and right to deploy the new model on their site. The new model measured all the old things that agencies have been measuring online with us for 10 years, it just added these new elements so that agencies could evaluate their success on these open government initiatives and report their success if they choose.
The E-Government Transparency Index
We decided to publish a report with scores for the 14 sites that had enough data collected by the end of December. We have high standards for data collection, so while many more sites are measuring transparency, only 14 had a big enough sample by the end of the year. Voila, The E-Gov Tranparency Index. Kudos to all that let us publish their scores, and we are looking forward to seeing the Index grow and expand in the coming year.
Just in time!
Last December, the White House and OMB released another memo setting an aggressive time line for agencies to be more open and solicit public opinions. Those of you who have the new model implemented on your site should be well on your way to quantifying how transparent your website is with ACSI-modeled scores for trust and transparency, which are benchmarkable across agencies. Those of you who don't have the new model implemented, just contact your SRA and they can fix you up. Those of you who are not currently running the ACSI survey online can contact dave.lewan@foreseeresults.com and we'll help you out.
So what do you think? Any surprises? Who would you like to see included in the Transparency Index next time?