Introduction to Product Analytics

2026 Download Slides

This is the first part of a series on product analytics and A/B testing. Most of the examples come from online businesses, because digital products make the ideas easy to see, but everything here carries over to a brick-and-mortar store, a marketing team, or even a policy decision.

Before we can talk about how to run an A/B test, we need to understand why we would want to. That "why" lives in product analytics, so that's where we start.

What Is Product Analytics?

Product analytics is the science (and a little bit the art) of using data to understand and improve how people experience a product.

Think about the apps you use every day: Netflix, Spotify, Amazon. Every time you click, search, or add something to your cart, the company learns something about what you like and how you use their product. That's product analytics in action.

Spotify, for example, tracks things like:

Which songs people skip. A skip suggests you don't like that style or composition, so Spotify can steer future recommendations away from it.
How long people listen to different types of music. That tells them what you actually enjoy, not just what you clicked once.
What time of day people listen to certain genres. So they can predict what you'll want to hear at 7 a.m. versus 11 p.m.

A food delivery app might look at:

How long people spend looking at menus.
The point where users abandon their cart. Maybe the delivery fee shows up after everything is added, and it's too high.
Restaurants that get lots of clicks but few orders. That could be a pricing problem, or the food just doesn't look appealing in the photos.

The key idea: these companies aren't improving their products by guesswork. They are systematically collecting and analyzing data from how the product is actually used.

The three goals

Why collect all this data? There are three core objectives, and they build on each other.

Understand user behavior. Every click, scroll, and tap leaves a digital footprint. Those footprints show which features users love, where they get confused or drop off, and when they're most active.
Improve engagement. Think of a restaurant: if customers love your Tuesday taco special, you might add more special days. Online, the questions are the same. What keeps users coming back? Why do some become superfans while others leave? What would make the experience good enough that people tell their friends?
Optimize business outcomes. In the end, a business has to be profitable. Are users paying for premium features? Which marketing campaigns bring in loyal users? Where are we spending money without seeing results?

When you understand users, you can make the product more engaging, and as engagement improves, business outcomes improve. Everybody wins.

The Four Components of Product Analytics

Product analytics runs as a loop with four steps.

Data collection: capturing user interactions. Think of it as listening to your users, except they're speaking through clicks, scrolls, video plays, and button taps instead of words. A fitness app might track how often people log workouts, which routines they finish, and whether they set goals or just browse. The important thing is to collect this data consistently.
Data analysis: finding patterns and trends. Now you sort through all those clicks and views. Maybe users log fewer workouts on Mondays and more on weekends. Maybe an online store sees lots of items added to carts but few checkouts. Breaking the data down, for example by user group, shows where things work and where they don't.
Insight generation: turning patterns into something you can act on. Patterns by themselves aren't enough. If a streaming service sees that users often pause during specific episodes, the insight might be that those episodes are too long or not engaging enough.
Decision-making: putting insights into action. Say a ride-sharing app discovers that people cancel when wait times go over seven minutes. They might offer a discount for longer waits, move drivers to high-demand areas, or send a notification when wait times drop.

Then the loop starts again. Every decision produces new data, which leads to new insights and further improvements.

Why Product Analytics Matters

Imagine driving a car without a dashboard: no speedometer, no fuel gauge, no GPS. You'd be driving blind. A business without product analytics is doing exactly that, making decisions with no real data behind them.

Product analytics helps in four ways:

It predicts future performance. Trends today hint at what's coming. A drop in daily logins can be an early warning that engagement is slipping, and spotting it early means you can act before too many users leave.
It drives product improvements. It shows you exactly where users struggle, which features they love, and what needs fixing.
It aligns teams. Product managers, marketers, and customer support all look at the same data, so it's easier to agree on what's working and what needs to change.
It provides a competitive advantage. Companies that decide with data move faster and build the features users actually want, while competitors are still guessing.

Use Cases

Here's how product analytics shows up across different products.

Customer retention: why do some users stay while others leave? On a subscription service like Netflix, some people cancel after a month and others stay for years. Analytics helps you find what keeps loyal customers engaged, maybe the personalized recommendations or the steady stream of binge-worthy shows, so you can do more of it.
Feature usage: which features are hits and which are ignored? Companies add features all the time, but are people using them? If a new Instagram filter isn't getting traction, they might tweak it or scrap it.
"Aha" moments: when does a user realize the product is valuable? On Facebook, it was reconnecting with old friends. On Spotify, it's discovering a new favorite song in a personalized playlist. On Netflix, it's catching yourself binge-watching a recommended show. Once you know what triggers that moment, you can help new users get there faster, which boosts long-term engagement.
Revenue growth: how do product changes affect sales or subscriptions? Maybe you learn that longer free trials lead to more long-term subscribers. Analytics tells you whether a change was a hit or a miss.

Where Product Analytics Fits

Product analytics is one piece of a bigger analytics picture. Think of a sports team with an offense coach, a defense coach, a physio, and so on. Each focuses on their own area, and they all work together to win.

Marketing analytics: how do users find us? Which ads or campaigns drive the most traffic? Which channels (social media, search, referrals) bring in the best users? How much does it cost to acquire a new user (Customer Acquisition Cost, or CAC)?
Customer analytics: what's the bigger picture of the relationship? How often do people engage with the brand, not just the product? What is a user's Customer Lifetime Value (CLV), the revenue they generate over time? Why do some people stop engaging or unsubscribe (churn)? This includes things like support interactions and brand loyalty.
Product analytics: is the experience good? How do users move through the product, which features do they use, where do they get stuck, and what changes would improve engagement and retention?
Business analytics: the big-picture strategy. How much revenue are we making overall? Where should we put resources to grow? What's the return on investment (ROI) of our strategies? Business analytics pulls from the other three.

Here's how they connect. You launch a new feature. Marketing analytics says it brought in lots of new users. Product analytics shows only a few of them are actually using the feature. Customer analytics shows that the ones who do use it become long-term customers. Business analytics then checks whether overall revenue and retention improved.

Skills of a Good Product Analyst

Curiosity. Great analysts don't take numbers at face value; they ask why. If people add items to their cart but never check out, a curious analyst checks whether shipping fees are too high, the checkout is confusing, or something else is off.
Storytelling with data. Numbers mean nothing unless people understand them. Instead of "engagement dropped 10%," say "engagement dropped 10% after we changed the homepage layout, which suggests users are struggling to find key features."
Technical skills. You don't need to be an engineer, but you should be comfortable with SQL for querying databases, Python or R for deeper analysis, and a visualization tool like Tableau, Looker, or Power BI.
Business acumen. Know what the business is trying to do. An analyst at a streaming service should know whether the company is focused on retention, growth, or monetization, and prioritize analyses that serve that goal.
Collaboration. You'll work with product managers to prioritize features, with marketing to refine campaigns, and with engineering to make sure data is tracked correctly. If a feature is underused, you'll need designers and engineers to figure out why.

Metrics: The Clues

Think of product analytics as a detective solving a mystery, and metrics as the clues.

Metrics are the specific numbers we track: Daily Active Users, average listening time, skip rate, playlist creation rate. They tell you what is happening.
Product analytics is the detective's process. It gathers the clues, looks for patterns, and asks "why?" and "so what?"

The process goes: choose the metrics that matter, collect them consistently, then analyze them together. For example, Daily Active Users drops. Digging in, you find session lengths are shorter, error rates are higher, and usage of a key feature fell. Connect the dots and it turns out a recent update introduced bugs, so people spend less time in the app.

Metrics tell you what is happening. Product analytics explains why, and what to do about it.

Why metrics matter

Running a business without metrics is like driving with your eyes closed. Metrics are tied directly to your business model: where revenue comes from, what things cost, how many customers you have, and how well you acquire new ones.

Instead of just feeling that users love a Netflix show, metrics let you see how many people watch it each day, whether they finish episodes, whether they rewatch, and how they rate it. Metrics turn gut feelings into facts, and when something goes wrong, you spot it right away instead of weeks later.

Three common examples:

Retention rate. Think of a gym. If people sign up and stop coming after a month, something is off. High retention means people keep coming back.
Conversion rate. A site can get tons of visitors, but if few of them sign up or buy, the conversion rate is low and the experience probably needs work.
Cost per acquisition (CPA). Spending $50 in ads to win a customer who only spends $10 isn't sustainable. Lower CPA means you're acquiring customers efficiently.

Ratios: Putting Numbers in Context

One number rarely tells the whole story. A friend says they made 100 sales last month. Impressive! Then they mention they spent $10,000 on advertising to get them. Divide one by the other, and each sale cost $100 in ads. Suddenly 100 sales doesn't sound as good.

Or take walking. "I walked 2 miles" doesn't say much. "I walked 2 miles in 30 minutes" tells you my pace, and you can compare it to yours.

That's what ratios do: they compare two related numbers so you see the full picture. Common business ratios:

Ratio	Example	What it tells you
Conversion rate	50 purchases ÷ 1,000 visitors = 5%	How well you turn visitors into customers
Cost per acquisition	$500 marketing ÷ 10 new customers = $50	How efficient your marketing is
Revenue per user	$1,000 revenue ÷ 100 users = $10	How much value each user brings

Why prefer ratios?

They account for size. A small company with 100 sales from 1,000 visitors (10% conversion) may be doing better than a big company with 1,000 sales from 20,000 visitors (5%).
They make targets simpler. Instead of juggling "increase sales" and "reduce spending," you can focus on one goal: "reduce cost per sale."
They make comparisons fair, over time and against competitors.

Setting up a ratio

Most ratios compare what you put in (money, time, people, materials) with what you get out (sales, happy customers, problems solved). If a coffee shop had 1,000 customers and $5,000 in sales, dividing sales by customers tells you the average customer spends $5. Now that's useful.

Ratios usually come in two flavors:

Efficiency metrics, where lower is better. Cost per customer, time per task. "How much do we spend to get one good result?" Put the cost on top: $10 per customer.
Success metrics, where higher is better. Conversion rate, satisfaction score. "How often do we succeed?" Put the total (like total visitors) on the bottom to get a percentage: a 10% conversion rate.

Don't forget absolute numbers

Ratios are great for efficiency, but raw totals matter too.

Scale matters. A 5% conversion rate on 1,000 visitors is 50 customers. The same 5% on 100,000 visitors is 5,000.
Planning needs totals. Knowing you gained 10,000 new users or made $100,000 in sales is what you budget against.
Absolute numbers show size; ratios show efficiency. Use absolute metrics for market size, growth, and adoption, and use ratios to benchmark performance. The best picture uses both.

What Makes a Good Metric?

Think about the score in a video game. A good score lets you see whether you're improving, how you rank against friends, and how close you are to the high score. A good business metric does the same thing. It is:

Comparable. You can compare it over time, across user groups, or against competitors.
Understandable. Everyone on the team knows what it means. If it's too complicated, nobody acts on it.
Reflective of trade-offs. Businesses juggle: more customers without overspending, more revenue per customer without pricing people out, higher quality without slowing down. "100 sales" tells you less than "100 sales from 10,000 visitors," which is a 1% conversion rate.
Actionable. This is the most important one. When the metric moves, you should know what to do. A fitness tracker works this way: if you're behind on steps, you know to get up and walk.

Metrics work in groups

When you drive, you don't just watch your speed. You also check fuel, engine temperature, and warning lights. Business metrics are the same: one number alone can mislead.

A useful way to balance them is the Speed–Quality–Cost triangle. For an online store:

Speed: How quickly do customers make a purchase?
Quality: How many visitors actually buy (conversion rate)?
Cost: How much do you spend to get each new customer?

If sales are happening fast but you're barely making a profit, is that good? Would you rather have fewer sales with bigger margins? Tracking all three together helps you answer that.

Netflix doesn't only count new sign-ups. It also tracks churn (how many cancel), daily active users (how often people watch), and watch time (how long they stay). Pairs of metrics raise better questions:

Fast checkout (2 minutes on average) + conversion rate (3.2%): does speed actually lead to more sales?
Cart abandonment (65%) + page load time (3 seconds): is slow loading driving people away?

Five Questions for Choosing Metrics

When you're deciding what to track, run each candidate metric through five questions.

1. Quantitative or qualitative?

Quantitative metrics are numbers; they tell you what is happening. Qualitative metrics are descriptions; they tell you why.

Imagine your favorite restaurant is struggling. The numbers say:

Only 50 customers came in today (usually 100).
Average spending per table is $45 (down from $60).
30% of customers never come back (up from 10%).

If you only look at those, you might decide, "People are spending less, let's raise prices!" But talk to customers and you hear: "The new menu is confusing." "The service has gotten slow." "The food isn't as good as it used to be." It turns out people spend less because service is slow and they leave before dessert. Raising prices would make it worse.

Same with an app that gets 1,000 downloads a day, where 70% of people delete it within 24 hours. The download number looks great. The feedback says: "I couldn't figure out how to create an account." "The tutorial was too long." "It kept crashing." Use both kinds of data together.

2. Vanity or actionable?

If you're trying to get in shape, "I've been to the gym 100 times" sounds impressive, but it doesn't tell you whether you're getting stronger. "I can do 10 more push-ups than last month" does. The first is a vanity metric; the second is actionable.

Vanity (looks good)	Actionable (drives a decision)
100,000 app downloads	30% of people who download are still using the app a week later
1 million followers	Only 2% of followers click our links
50,000 website visits	5% of visitors make a purchase

A game studio might say, "We got 500,000 downloads last month!" The actionable version is: "50% of new players quit on day one, mostly at level 3. We need to make level 3 easier."

The test is simple: if this number changes, do I know what to do? If yes, it's actionable. If not, it might just be vanity.

3. Exploratory or reporting?

Reporting metrics are your regular health check: daily revenue, active users, conversion rate, revenue per user. You already know they matter, so you check them on a schedule and compare over time.

Exploratory metrics are detective work. You dig into the data without a specific question to see what surprises turn up.

The classic story is Facebook's "Circle of Friends" app, which let people group their friends. It had around 10 million users, but fewer than 20% of the groups were actually active. Looking closer, the team found that moms were the power users: they wrote longer messages, shared more photos, and started more conversations. So the app pivoted to "Circle of Moms." The total user count dropped, but engagement soared, and the company was eventually acquired.

You need both. Reporting metrics are the map of where you know you need to go. Exploratory analysis is the compass that points to opportunities you didn't know existed. In practice:

Set up a dashboard for the metrics you know matter.
Set aside time each month to explore the data.
When something interesting appears, dig in with user interviews or controlled tests.
Be willing to act on what you find, even if it means changing strategy.

4. Leading or lagging?

Leading metrics are early warnings, like a weather forecast. Lagging metrics are the report card; they tell you how things turned out.

For a restaurant, next week's reservations are a leading metric. If they're low, you can still run a promotion. Last month's revenue is lagging: it tells you how you did, but you can't change it.

In e-commerce, the add-to-cart rate is leading: if people aren't adding items, sales will probably be low. End-of-month sales are lagging and confirm what happened. Leading metrics let you act early; lagging metrics confirm whether it worked. You want both, like having a forecast and a thermometer.

5. Correlated or causal?

You've heard it before: correlation doesn't mean causation. Ice cream sales and swimming pool accidents both go up in summer. Ice cream doesn't cause accidents; both are driven by the heat.

In business, a company notices that website traffic rises when its ad budget rises. Is the ad spend causing the traffic? Maybe not. Other things could be happening at the same time:

Seasonal trends, like holiday shopping.
A competitor cutting their ads, making yours more visible.
Organic growth from word of mouth.

Causation is when changing one thing directly changes another. Make the "Buy Now" button bigger and brighter, and if clicks and purchases go up as a result, that's cause and effect.

Correlation can help you predict, but it doesn't tell you what to do. Causation tells you which lever to pull. The practical way to establish causation is an A/B test: change one thing, keep everything else the same, split traffic between the old and new versions, and compare. That's what the rest of this series builds toward.

KPIs: Your Business's Vital Signs

Key Performance Indicators (KPIs) are the handful of metrics that matter most. The five questions above work as a filter: run every candidate metric through them, and the ones that come out quantitative, actionable, and predictive (or clearly tied to outcomes you care about) are your KPI candidates.

Not every metric you track is a KPI. KPIs:

reflect progress toward your most important goals,
tell you whether you're actually delivering value,
tell you when it's time to change course, and
keep everyone focused, because there are only a few of them.

At the top sits your North Star Metric: the single number that best captures the core value your product delivers. It's the guiding light every team aligns around. We'll build one step by step in Measuring Success.

Two examples:

Restaurant. The primary KPI is the number of tables served per night ("covers"). You might also track phone inquiries or popular dishes, but covers are what reflect success.
Subscription app. KPIs might include Monthly Recurring Revenue (MRR) and churn rate, with active engagement as the North Star, because that behavior drives long-term growth.

One last check: if a metric changes and you know exactly what to do next, it can be a KPI. If not, it may look good on paper without driving any real value.

Key Takeaways

Product analytics uses data about how people use a product to understand them, keep them engaged, and grow the business.
It runs in a loop: collect, analyze, turn patterns into insights, decide, and repeat.
Metrics tell you what is happening; analytics explains why and what to do next.
Ratios add context and absolute numbers add scale. Use both.
A good metric is comparable, understandable, reflects trade-offs, and above all is actionable.
Check each metric with five questions: quantitative or qualitative, vanity or actionable, exploratory or reporting, leading or lagging, correlated or causal.
Your KPIs are the few metrics that survive that filter, with a North Star Metric at the top.
Only a controlled experiment, like an A/B test, can tell you that a change caused a result.

Next in the series: Analyzing Declining Metrics