User Analytics

Launching a new product or experience is exciting, especially if you expect that it will boost you business in a substantial way. But after you’ve launched, it’s time to assess how users are actually responding to what you've put out there, in real life.

How long are visitors staying on your site or in your app? How long does it take for them to complete key tasks? Are they stumbling on the same things over and over again? Are they buying the stuff you want them to buy? The stuff they indicated they would buy in your previous research? Are they generally picking up what you’re putting down?

Many of these questions can be answered by reviewing user and web analytics.

Start with user goals

Begin with understanding your user’s goals. For example, if you’ve created an experience within a fitness app for someone who wants to get lean, you want to understand which actions are the most important for meeting this goal, then track the ease with which folks can complete those actions.

So, how do you know when someone’s having trouble? Sometimes users give you feedback directly through a survey or support interaction. Other times the writing is on the wall in the form of quantitative analytics—like Google Analytics for web, Mixpanel for product events, or other tools in your analytics stack.

Understand why reviewing analytics matters

Quantitative analytics are important to all organizations, with different departments and functions monitoring what is most relevant to their needs. The C-Suite is likely watching top line metrics, like daily, weekly, and monthly revenue, plus whatever key metrics drive that revenue. Marketers are focused on metrics that drive revenue throughout the entire funnel. Product teams are focused on product usage and user centric metrics that help drive that usage, and ultimately revenue.

Whoever the person or team, the beauty of rallying around quantitative metrics is everyone can speak the same language in a pretty objective way. In the context of ongoing listening following a product release, you probably have some historical benchmarks to watch post launch, focusing in on the areas you were seeking to impact. Mapping user goals, to product goals, to revenue goals, to quantitative metrics is an important part of aligning goals across an organization.

Determine the essential analytics you want to measure

Depending on the goals of your product launch or feature updates, you may want to focus on some of the below common quantitative metrics.

Survey analytics

• NPS, CSAT, or CES scores - As outlined in Chapter 1, these survey metrics give you an idea of how your customers feel about your company and particular touchpoints with your company, such as the support experience. If these scores are changing (positively or negatively) following a launch, this can be an indication of how your changes are being received. Drilling into your data with a focus on key segments and cohorts, and validating with further data will help you uncover insights.
• Qualitative survey data - Often the above surveys include a free-form question that can help you understand some of the whys, the motivations behind the positive or negative scores. As you combine quantitative data with qualitative and segment it by meaningful customer groups, you should start to from hypothesis you can then validate through testing and further research.

Product analytics

• Feature use - Which features are used and the most? Are your new features, or updates, getting used?
• Recency and frequency - How recently did someone use your product? How frequently do they return?
• Value of use - Are people who use a given feature more valuable, happy, or otherwise positively impacted by using it?

Web analytics

• Time on site - This can be a great area to drill into, especially for a content driven experience, or one where the revenue model is closely tied to visit duration.
• Visits - How many people visited your experience overall? How are visits changing over time? More isn’t always more, but all things being equal, it is.
• Unique visitors - How many different users are interacting with the experience?
• Goal completion - From leads generated, to purchases completed, to buttons clicked, if you can tag it, you can track it.
• Pages visited - Which pages did an individual or group of individuals visit? Are those pages connected to key buyer/user journeys? 
• Traffic source - How did the user enter your experience? Was it from a certain campaign, organic traffic, or another source?
• Path to conversion - Before converting, which pages do users visit? How long do they stay on site? Is this path as direct as possible? Is there an opportunity to improve your site navigation? Evaluate across different personas, lifecycle stages, or user stories. Not every user has the same goals.

By understanding and tracking your baseline against the metrics that matter to your goals, and your users goals, you’ll quickly notice where a product update or launch is having an impact.

Set up a system for measuring analytics

Once you've determined the metrics you need to track, find what teams may already be using, then what you may need and don't have. Some user researchers build out custom dashboards, integrating Google Analytics with Microsoft Excel, or taking advantage of one of the many business intelligence tools now available. Whatever you can do to automate your dashboards so you can spend less time collecting your data, and more time analyzing and acting on it, is great. Make sure to share your dashboards with other stakeholders too!

A good dashboard:

• Prominently highlights the most important data
• Illustrates change, anomalies, data worth noticing
• Is connected to deeper data for drilling in where necessary to understand what’s “really going on.”

Create a cadence for reviewing analytics

Perhaps you have a weekly department meeting where key metrics review is a recurring agenda item. Or maybe you have a monthly OKR meeting. Or maybe you are the lead researcher on a particular project you’re very invested in, and you need to know what’s happening by the day or hour on something that has just launched. Depending on your individual, team, or company situation, set up a regular cadence for reviewing the metrics that matter to you the most. Of course, it may make sense to review different metrics at a different cadence. We recommend a weekly check-in of your top level key metrics at a minimum, and then build from there based on your needs. Give yourself a recurring task or calendar event to make this ongoing review a habit or take advantage of automated email reports for your services that offer them.