On Engagement

Eric Peterson, in his quest for meaningful metrics, has laid out his proposal for a comprehensive engagement metric.

I love the effort in the pursuit of the “best” metrics, hopefully helping make our jobs of web analytics not so hard. There are some very interesting components included in Eric’s visitor engagement metric, such as the brand component. But I still have some skepticism:

First, I’m not sure I really see what this tells me as a site owner or how to make this actionable. Because engagement, as calculated according to Eric’s visitor engagement formula can be good or bad.

• Click depth:  If it takes me 10 clicks to find something I should have found in 3, that doesn’t mean I’m more engaged, just more frustrated and closer to taking my loyalty somewhere else.
• Duration: I find this very problematic because of the number of times I’ve spent quite a bit of time on a website only to leave frustrated. I don’t think it’s a good measure of how engaged I am.
• Feedback Index: just because people left feedback doesn’t mean they are engaged in a positive way. Many feedback mechanisms promote as much negative feedback as positive. While it is always good to get feedback, I need to evaluate positive feedback very differently then negative feedback.

Fundamentally, what does engagement really tell us? If I’m in charge of Target.com, I care about what people bought, how the website influenced in-store sales, etc. Engagement doesn’t really get at what I need to know because it doesn’t tell me what people will do next week or next month or how to influence that behavior.

My website objective might be to sell products or services, to inform, to use as a primary resource, to build the brand, etc. How many sites have engagement as a website objective? What does engagement tell me about my success in meeting those objectives?

In Eric’s case, I assume his primary objective is to sell consulting services, secondary is to sell books. And that the features of his website, like the blog and the job board, are there to increase likelihood of the first two things and increase the brand value of Eric. So he should care a lot more about how engaged potential clients are than how engaged job seekers are. And what about a long visit that results in a negative comment (although – I am sure that never happens on Eric’s site :) ).

And again, while I appreciate the attempt to make this comprehensive, I think by measuring this many things and giving them equal weight, what you end up with is a watered down metric that will not be able to predict how well a site will meet the kinds of key objectives I mentioned above.

Bottom line: While a step forward, the visitor engagement calculation will not measure success. Success is: "did our users accomplish what they wanted?" and "were they satisfied?"

Comments

[Eric T. Peterson]

Larry,

Thanks for referring to my work on visitor engagement, and thanks again for your comment in my blog. I guess I personally think that if you're trying to measure something simple like conversion, or even something moderately complex like customer satisfaction, you're all set with many of the metrics in widespread deployment today.

But I guess my work on engagement is designed to address all the people that aren't converting, that aren't yet convinced to purchase in store, and aren't really satisfied or unsatisfied. Other folks have postulated that more traditional metrics like "average time spent" and "page views per session" are useful in these situations -- I don't disagree, but I don't think we should shy away from new ideas when they can be shown to be useful.

As I commented back to you in my blog, I see visitor engagement as being complimentary to conversion and customer satisfaction. Not better, not worse, complimentary. Only the test of time will tell if this metric has the predictive power of the ACSI or the raw utility of conversion.

I think it will. But I also think there are thousands of sites out there focused on finding and satisfying "engaged" visitors. Facebook comes to mind.

Nick Arnett has questioned whether I have "ground truth" for my measure. Can I put you down as "engaged" yet?

Eric T. Peterson
Web Analytics Demystified, Inc.
http://www.webanalyticsdemystified.com/

[Steve Jackson]

Hi Larry,

I would like to answer this question because it reflects a debate I recently had with Avinash around the subject.

It all boils down to semantics. You ask;

"Fundamentally, what does engagement really tell us?"

The answer to that question is depends on what you call engagement. I call engagement the degree and depth of your visitors interaction with your online presence and the way you distinguish an engaged visitor from a casual visitor. You are calling it sales (something I would categorize as activation).

So semantically we're talking about different things. What I would use Eric's engagement indices for is to find out for instance if there really were usability problems or it was people happily reading. If there was 10 clicks and it indicated potential confusion I would then do some qualitative assessment as to why. A heuristic analysis at least.

Br
Steve.

[Larry Freed]

Steve,
Thanks for the comment. Your definition of engagement ("the degree and dept of your visitor interaction..."), is good, and I feel represents well what Eric's engagement calculation will tell you. My question centers around, is that engagement good? So, if engagement doesn't distinguish positive image of the site from a negative image, does it matter how engaged they were. Now, on a blog, you could say that engagement is the measure of success. But what about informational sites (Webmd, Medline plus) or news sites (CNN, Wall St. Journal)? Is engagement there good? Is a long time on the site good or bad? Depends on if you found what you were looking for. Now it might be good for your advertisers, but is it good for your visitors. If you spend 10 minutes, 20 pages looking for an article on WSJ and don't find it, you may have seen a lot of ads, but you may never come back because you are a dissatisfied customer. From my perspective, I want to spend as little time on those sites as possible and still find what I am looking for. That is what makes me satisfied and loyal.

So, my concerns with engagement are similar to my concerns with page views, duration of site, etc., being used as metrics, that are often taken to be measures of success.

Thanks again for the comment.
-Larry

[Matthew Coen]

Larry,

Though your points regarding engagement are not without merit, the perspective from which you address the topic it too myopic.

If you think in terms of managing all aspect of a web business (media for example) the importance of engagement will become more obvious.

Regards,

Matthew Coen
Manager, Business Intelligence
Nissan North America, Inc.

[Larry Freed]

Matt,
It is good to hear from you. It has been a long time. I hope all is going well for you.

I agree that as you look at other websites, such as media, the role of engagement becomes more important. From an ad-revenue perspective, the more time on the site, the better! But does the engagement index complicate things for that purpose? If a user makes a comment on the site, does that really matter from an ad-revenue perspective? And if you think it does, does it matter if it was a positive or negative comment?

I think the effort to find the right way to measure engagement (especially for purposes of ad-revenue sites) can be important. But generally, I would say that being engaged with a site is much more positive if the user is engaged in a positive fashion - they are getting their needs met. As opposed to an angry visitor, who is leaving negative comments and is very frustrated because they cannot find what they are looking for -- but are diligent in their efforts.

Thanks for the comment.
-Larry