The metric that quietly changes meaning
Here’s a trap I fell into more than once: I’d calculate churn in January, feel good or bad about the number, and move on. Then in April I’d calculate it again, and the number would jump around in a way that didn’t match what I remembered actually happening in the business. Nobody canceled in some dramatic wave. What changed was how I was counting.
Maybe in January I excluded free trial cancellations. By April I’d forgotten I was doing that, or a new spreadsheet formula included them by accident. Maybe I switched from “customers who canceled this month” to “customers who canceled and didn’t come back within 30 days.” Small shifts, but they mean January’s number and April’s number aren’t actually comparable. I was comparing two different metrics that happened to share a name.
This is the core problem with most solo-operator reporting. It’s not that the metrics are wrong on any given day. It’s that they’re not built to survive contact with your future self, who won’t remember the exact rules you used, or a future tool migration, or a future “let me just tweak this real quick” moment. A metric you can only compute correctly once isn’t a metric. It’s a one-time calculation wearing a metric’s name.
What “repeatable” actually requires
A repeatable metric needs four things nailed down before you ever run the number, not after you’ve already got a result you like.
A precise definition. Not “monthly recurring revenue” but “the sum of active subscription plan values, in USD, as of the last day of the calendar month, excluding one-time charges and excluding invoices still in a failed-payment retry state.” That’s specific enough that two different people, or the same person six months apart, would get the same answer from the same data.
A fixed data source. If churn comes from your billing platform’s subscription status field this quarter, it needs to come from that same field next quarter, not from a support ticket tag that says “wants to cancel,” which measures intent, not action.
Clear time boundaries. Does a customer who cancels on the 31st and signs back up on the 2nd count as churned? Does “this month” mean calendar month, or a rolling 30 days from signup date? Pick one and write it down, because both are defensible and only one of them is what you’re actually using.
Explicit exclusions. Test accounts, comped accounts, refunded orders, internal team seats. Every business has a handful of records that shouldn’t be in the denominator, and if you don’t list them out, whether they’re included depends on whoever happens to be running the query that day.
None of this is exciting work. It’s the accounting equivalent of labeling your boxes before you move house. But skip it and every comparison you make later is built on sand.
A worked example: churn, three ways
Say you run a small subscription box business. You want to know your monthly churn rate. Here are three defensible ways to calculate it:
- Customers who canceled during the month, divided by customers active at the start of the month.
- Customers who canceled during the month, divided by the average of customers active at the start and end of the month.
- Revenue lost to cancellations during the month, divided by revenue at the start of the month (this is dollar churn, a different metric entirely, even though people often call both “churn”).
All three are legitimate. None is more “correct” than the others in the abstract. But they produce different numbers, and if you use method one in Q1 and switch to method two in Q2 because a course or a spreadsheet template nudged you that way, you’ll see a shift in your churn rate that has nothing to do with your actual customers and everything to do with your arithmetic. You’ll spend an afternoon worrying about a problem that isn’t real, or worse, you’ll miss a problem that is real because it’s buried under a definitional shift you didn’t notice.
The fix isn’t picking the “right” formula. It’s picking one, writing down which one and why, and using it every single time until you have a deliberate reason to change it, at which point you recalculate history under the old and new definitions side by side so you can see what actually moved and what’s just a bookkeeping artifact.
Write the definition down somewhere you’ll actually check
I keep a plain text file, one per metric, that answers four questions: what counts, what’s excluded, where the data comes from, and what date logic applies. For MRR it’s maybe eight lines. For customer lifetime value, which is more prone to drifting definitions (trailing 12 months of actual revenue? projected value based on average tenure? gross or net of refunds?), it’s more like fifteen.
This isn’t a big governance project. It’s closer to a recipe card. The point is that when I sit down next quarter to run the number again, I’m not reconstructing my reasoning from memory or from whatever the last spreadsheet happened to do. I’m reading the rule and applying it. That’s the whole trick. Repeatability doesn’t come from a clever tool. It comes from writing the boring decision down once so you don’t have to remake it, slightly differently, every time.
If you’re using a script or an automated dashboard to pull these numbers, the definition lives in the code or query, which helps, but only if you actually read that code before trusting its output rather than assuming it still does what you think it does. Tools drift too. A billing platform can quietly change what “active” means in its API. A spreadsheet formula can get edited by a collaborator, or by you, at 11pm, without a record of what changed.
What changes anyway, and how to handle it
Your data sources will change. You’ll switch billing platforms, add a new product line, or start excluding a category of customer you used to include. When that happens, don’t silently let the new definition bleed backward into old numbers, and don’t pretend nothing changed either. Recalculate a few historical periods under the new definition, note the date the definition changed, and keep both versions visible if the shift is large enough to matter. A metric with a documented break in it is still useful. A metric with an undocumented break in it just looks like a mystery trend six months later, and you’ll waste real time chasing a cause that doesn’t exist.
This is also where cohort-based analysis earns its keep. Instead of comparing “this month’s churn” to “last month’s churn” as single blended numbers, group customers by signup month and track each cohort’s behavior over time. Cohorts are more resistant to definitional drift because you’re comparing like groups to like groups, and a change in how you calculate things tends to show up as a visible seam rather than getting smeared across your whole history.
What repeatability can’t do for you
Being able to compute a metric the same way every quarter doesn’t make it the right metric, and it doesn’t make the number good news. A perfectly consistent churn calculation can still be telling you something uncomfortable, and consistency won’t soften that. It also won’t catch a bad underlying assumption, like counting downgrades as retained customers when really they’re a warning sign you’re choosing not to see. Repeatability buys you one specific thing: the ability to trust that a change in the number reflects a change in your business, not a change in your math. That’s worth a lot on its own, but it’s a floor, not a strategy.
If you want more on how I actually build and track these numbers month to month, including the specific definitions I use for churn, MRR, and CLV in my own businesses, you can find the rest of the writing and videos on the home page.
Get new guides and videos first — join the Telegram channel.