Why the baseline matters more than the metric
Every dashboard I have ever built starts with the same argument: what are we comparing this number to. Not what the number is. What it is measured against.
A conversion rate of 2.3% means nothing on its own. Is that good or bad? You cannot say until you pick something to hold it against, last month, last year, a competitor’s public numbers, or the average across your own product lines. That comparison point is your baseline, and it does more work than any formula in your analytics stack. Get the baseline wrong and every chart built on top of it is wrong too, no matter how clean the underlying data is.
I run a handful of small businesses off numbers like this, and the question I get asked most often is not “which tool should I use” or “how do I calculate churn.” It is “how do I know if this number is actually good.” That question can only be answered with a baseline, and it has to be one you can explain and defend when someone (a partner, an investor, or just yourself three months from now) asks why you picked it.
What “defend” actually means
A baseline you can defend is one where you can answer three questions without hand-waving:
Where did this number come from. Not “roughly what I remember” but a specific period, a specific data pull, a specific definition of what counts.
Why is this the right comparison. Why last quarter and not last year. Why your own history and not an industry benchmark.
What would make you throw it out. If nothing would change your mind about the baseline, it is not a baseline, it is an assumption wearing a baseline’s clothes.
Most people fail on the third one. They pick a number that feels reasonable, anchor every future decision to it, and never revisit whether the anchor still makes sense. That is how you end up celebrating a “40% improvement” over a baseline that was broken from the start.
The three baselines people reach for first, and why they fall apart
The industry average. You read that “good SaaS churn is under 5% monthly” somewhere and adopt it as your target. The problem is that industry averages blend businesses with wildly different price points, contract lengths, and customer types. A $9/month tool sold to individuals and a $900/month tool sold to companies do not churn the same way, and neither should be judged against a blended number pulled from neither. Industry figures are useful for a gut check on whether you are in a reasonable range. They are a poor substitute for your own baseline because you cannot verify how they were calculated or who is in the sample.
Last period, always. Comparing this month to last month is easy to set up and easy to misread. Almost every small business has a seasonal or cyclical pattern, a slow week around a holiday, a spike after a promotion, a dip when a big customer pauses. If your baseline is always “last month” you will chase noise. A drop from a promotional spike back to a normal week looks like a crisis on a month-over-month chart. It is not. It is regression to a mean you never defined.
Whatever number you started at. Founders anchor on day one numbers without meaning to. “We used to convert at 4%, now we’re at 2.5%, something’s broken.” Maybe. Or maybe the 4% came from your first fifty visitors, who were mostly people you personally invited, and it was never a representative number to begin with. A baseline set before you had enough volume to be stable is not a baseline, it is a coincidence.
None of these are wrong to look at. They are wrong to treat as the single number everything else gets judged against, without asking whether they hold up.
Building a baseline you can actually justify
The baselines that survive scrutiny share a few traits.
They cover enough time to smooth out noise. For a metric that moves daily, like conversion rate on a small site, a rolling average over several weeks tells you more than any single day. For a metric that moves monthly, like churn or MRR, you generally want a few cycles before you trust the pattern, because one month can be distorted by a single large customer leaving or a billing glitch.
They match the thing you are actually trying to learn. If you are testing a new pricing page, your baseline should be the conversion rate of the old pricing page over a comparable traffic mix, not your site-wide average across every landing page you run. If you are checking whether a new onboarding flow reduces first-month churn, your baseline is the first-month churn of cohorts that went through the old flow, not your all-time churn number which mixes cohorts of every age.
They are documented somewhere other than your memory. A baseline you can defend is one you wrote down: the date range, the definition of the metric, and the reason you picked that range. If you cannot reconstruct why you chose a baseline, you cannot defend it when it is questioned, and you will not trust your own reasoning when the number moves.
They get revisited on a schedule, not just when something looks wrong. If you only reexamine your baseline when a number surprises you, you will unconsciously pick whichever old baseline makes the current result look better or worse depending on your mood that day. Set a fixed point, quarterly is reasonable for most small operations, to ask whether the comparison you are using still fits the business you actually run now.
A worked example: setting a churn baseline
Say you run a subscription tool and you want to know if a change to your cancellation flow, adding a short survey before the cancel button, actually reduces churn or just adds friction.
The wrong baseline is your all-time churn rate. That number blends customers who joined in your first month, when the product barely worked, with customers who joined last week under a completely different feature set and price. It is not a fair comparison for a flow-level change.
A defensible baseline is the churn rate of cohorts that went through checkout in a comparable recent window, say the three months before you shipped the change, filtered to the same plan tier the survey now appears on. You would track that cohort’s churn at the same point in their lifecycle (30 days, 60 days, 90 days) that you plan to measure the new cohort against, because churn is rarely flat over a customer’s life. Early cancellations and later cancellations often have different causes, and comparing month-three churn on your old cohort to month-one churn on your new cohort will make a boring, unchanged product look like it improved.
You would also write down, before you look at results, what change would actually move you: a few points of difference that holds up over more than one cohort, not a single month’s blip. Otherwise you will find a way to call whatever number shows up “meaningful.”
When to change your baseline
Baselines should move when the business genuinely changes underneath them. A pricing change, a new customer segment, a shift from self-serve to sales-assisted, these are all real reasons to reset. What is not a good reason is a baseline that has simply become inconvenient because it makes recent performance look flat or worse. If you catch yourself wanting to change the comparison point right after a bad result, that is exactly the moment to hold the line and ask why, honestly, before touching it.
What a baseline can’t do
A well chosen baseline tells you whether a number moved relative to something specific and defensible. It does not tell you why it moved, and it does not prove that the thing you changed caused the movement. That takes a proper test design, a large enough sample, and honesty about confounding factors like seasonality or a competitor’s launch that happened the same week. A baseline is the ruler, not the explanation. Treat it as one input into a judgment call, not a verdict, and it will serve you a lot longer than a number you picked once and stopped questioning.
If you want more on how to build the analysis around a baseline once you have one, cohorts, attribution, and the data quality checks that keep any of this honest, you can find the rest of what I write on this at Data Research Analysis Collection.
Get new guides and videos first — join the Telegram channel.