FUZZY NUMBERS: OBSERVATIONS ON MEASURING USER BEHAVIOR ONLINE

-

As much as we would like to have precise data on how users interact with devices, we are not there yet. This is not an in-depth study of analytics methodology but a statement about how I see web analytics used and some of the apples and oranges comparisons I have seen. 
To those that practice measuring user behavior online this may seem obvious. To those that rely on reports about user behavior online this may at first be an unsettling opening statement. I have recently come across a few instances where the definitions and nomenclature used by technical and marketing people vary in definition. This has led to some confusion as to what we are measuring, why we are measuring and how we measure.

The Business of Data Collection has Changed
Online behavior analytics has matured greatly over the past ten years. Architectures involving web services have also evolved from mostly 3-tiered models to n-tiered Serviced Based Architecture (SOA) models. In addition to the shift from dedicated servers to more distributed cloud models we also have rapidly evolving user interfaces (UI) that incorporate more complex code. These pages are not typical a then b then c pages but rather a single page with JavaScript (or whatever your code preference is) containers that load the data on the page making it very hard to track what the end user is doing.

Fuzzy Numbers: Uncertainly and the Observer Effect
Measuring user behavior online is not as easy as pasting a tag into a webpage. Google Analytics and WebTrends are two of the tools I use on a regular basis. Frequently I see both tags on the same page. This creates two problems seen mostly in quantum mechanics (QM): The uncertainty principle; The observer effect.

The uncertainty principle in QM really applied to particle physics and is also known as the Heisenberg Principle. Basically it states the more precisely we measure a particle, the more fuzzy measuring the momentum is and vice-versa. Where I see an analog in measuring online user behavior is, based on the technique used, the more we measure one aspect, the less we focus on other aspects. Analysis using a click-tracking service will certainly capture granular data about users that physical logs might miss but we will also be sampling users rather than capturing 100%. The reasons for this might seem obvious if we observe our own experience. I often use "incognito" mode in Chrome to avoid being bombarded by ads related to my browsing activity. I am fairly certain this works as I only get ads in content when I don't use incognito mode.

The observer effect seems much easier to explain. In using the same click-tracking methodology mentioned above we can make some assumptions: Adding code to a page increases latency; Adding links to external code for tracking or font APIs increases latency.

These two effects can be measured by simple sampling of pages that have tracking code and those that don't. One would hope that the latency effect would not be noticeable to the end user. Of course we need to use click-tracking as part of our tool kit when analyzing online user behavior but it should be seen as a tool for a larger purpose. 

Why are we Measuring Things?
This is one of the questions I am finding myself asking quite a bit. One of my favorite books in anything to do with the web is Jesse James Garrett's The Elements of User Experience. The illustration below is from one of the early chapters in the book and states pretty clearly that the first question in any web-based project should be "user needs and objectives". Is this something that is consistently a part of every engagement between web analysts and their business customers? In my fifteen years of experience the answer is "no".

Most of the time I deliver a report on the analytics of a page or pages and there are no question asking me why. I often offer an opinion backed up by using multiple sources including script based, alert based and log based. I am hoping to blog more specifically on my new adventures in measuring online user behavior. The market is mature and the tools are light years from where we started.

The fuzzy numbers we see are the result of the accuracy of our tools and the skills of the analyst to build a defensible methodology. All of this needs to begin with the question "why are we measuring this and what outcome do you, the stakeholder, want from the end user?"

More later...

Original article: https://www.linkedin.com/pulse/fuzzy-numbers-observations-measuring-user-behavior-hare-gaiq-apm-?trk=mp-reader-card

Comments

Post a Comment

Popular posts from this blog

Podcast: Open Data Discussions with Anthony Fung

-
Image
A look back at Virginia’s 4 years of data innovation with Anthony Fung, Deputy Secretary of Technology for the Commonwealth of Virginia  Welcome to the OpenDataSoft Leadership Podcast Series, “Open Data Discussions”. Each month, Jason Hare, our Open Data Evangelist, features a different open data program around the country to discuss what has made it successful. These examples will provide insights and strategies that you can implement in your own city. We had the pleasure of welcoming Anthony Fung, Deputy Secretary of Technology for the Commonwealth of Virginia for the fifth podcast in our series. Anthony discussed various subjects, including: What role does open data play in policymaking at the state level in Virginia? Does his department work on ensuring open data quality? What are some of the steps he makes in ensuring sound data governance? How did Virginia come up with this technology and engagement solutions?  How, for exampl...
Read more

WHITE HOUSE OPEN DATA INNOVATION SUMMIT - WHAT I SAID, WHAT I MEANT TO SAY

-
Image
+OpenDataSoft participated in the celebration that was 8 years in the making at the White House. Yesterday, for the first time, the White House held the 2016 Open Data Innovation Summit.  I work for OpenDataSoft as the Open Data Evangelist. I was asked to introduce the White House Agenda and then to speak on Civic Innovation regarding Open Data. What I said and what I wanted to say were different. The event was full of excitement and also of reflection Megan Smith and Tony Scott kicked off the event touting the 200k datasets that had been released through Data.Gov. Certainly this is refreshing from even a decade ago when serious effort and even lawsuits were required to make data available to the public. That all was not good and could be better was brought up by +Alex Howard from the Sunlight Foundation. Alex noted that HTML and PDFs were counted among the "datasets" being released. Note that while HTML and PDFs are indeed 1 star ...
Read more

Open Data Licensing

-
Image
Everything you wanted to know but you were afraid to ask. Making data open and available is not just about choosing a format or having an API. A poor choice in your license can prevent people from using them. A traditional open license reassures potential users that their work on the data will be useful in the long run. So, what license should you pick for your data to be broadly used? There are a few things to consider. Choosing a license for your open data can usually be one of the biggest challenges for an open data project. There are plenty of licenses available for your data. Choosing one mostly depends on the usage and reach you want to give to your project. CHOOSE A LICENSE: BASED ON NATIONAL, INTERNATIONAL AND LOCAL STATUTES IN ADDITION TO REUSE It should be noted that different parts of the world use data licensing terms differently. In the US, works in the “public domain” are works are not covered by intellectual property rights, such as copyright. The copyright mig...
Read more