How Sophisticated Are You?

I believe Stratigent’s The Evolution of Sophistication presented by Josh Manion was one of the most interesting things published in Web Analytics in recent months. In the paper, he shows the results of a survey conducted with more than 60 companies, covering their various types of practices in web Analytics as well as how they organize themselves. Josh stresses and examines how difficult it is to most organizations to evolve from a “basic web analytics program2 to an “advanced center of analysis”, in other words, the evolution of sophistication of Web Analytics.

Here are the 5 main conclusions of the report (I quote):

1. An evolution of Web Analytics practices exists
2. Pro-active data analysis is a defining characteristic of a sophisticated program
3. Education is critical for the development of sophistication
4. A large staff does not necessarily lead to sophistication
5. Organizational issues can impede progress

Amongst the most interesting findings of the research is the observed link between the existence of KPIs and whether an organization makes use of testing and scenario analysis or not.  The longer in terms of years you have worked with KPIs the more chances there are that you are conducting testing and scenario analysis. From my own experience, I don’t think KPIs adoption should strictly be understood by simply defining them; what you do with them, either or not you have made it possible for your KPIs to have an impact on how you manage your site, indicates true adoption.

The research regroups companies in four types. Let’s briefly review them:

• Basic Reporting: this is where one finds organizations that have just started in WA, or that have not progressed beyond implementing and using basic features. Those organizations have generally done very little customization, and still have difficulty building a knowledge base that they can share among stakeholders. Stratigent noticed that if a company stays at that level for more than a year, chances are that momentum around WA was not sufficiently built up, and that such companies run a real risk of getting stuck at that level.
• Advanced Reporting: This is where organizations will increase the level of sophistication of their tool use. This stage is however really reached when the question of KPIs is tackled. Having KPIs and conducting analyses based on them rapidly drives an organization’s needs towards conducting more pro-active data analysis.
• Advanced Analysis: This phase is mostly characterized by the use of more complex analytical techniques such as testing, advanced segmentation, and a more granular use of KPIs to meet the various stakeholders’s needs. At this stage, organizations start taking advantage of a continuous improvement process; multi-channel analysis integration is also put into place.
• Sustained Competitive Advantage: These are the most advanced organizations in Web Analytics. At that level, they make efficient use of all the available data, and are able to use WA ROI as a competitive advantage; “advanced multi-channel data integration”, “multi-channel segmented experimentation”, and “advanced life time value analysis” are their daily bread. These organizations have also solved the WA business process problem.

There are several other very interesting findings in this research paper I won’t comment here for post length purposes. Go get the paper; I guarantee you it is worth reading.

A few last words. Of course, the “Sustained Competitive Advantage” stage is the ideal situation, the El Dorado of Web Analytics. Personally, I have never come across one like that; I doubt they exist in Canada (correct me here if I’m wrong). This paper, the only one I know that has tried to categorize the evolution of the WA adoption, is to be read in relation with recent discussions about the impact of Web Analytics on business culture, or recent books such as the latest Thomas Davenport’s Competing on Analytics.

It is up to you to determine up to which stage you would like to bring your company. However, it is hard not to aim for the last stage, since it is more and more obvious that the biggest benefits of the longest term in WA reside there. I know, the road seems arduous and long, but at least we are all starting to see where we should end up someday. Start working!