The dashboard that hadn’t moved in three weeks
I once caught myself looking at a churn number that hadn’t changed in eighteen days. Not because churn had actually stopped, but because the scheduled job that pulled the number had failed on day one and just kept failing, silently, every morning after that. No error email. No red banner. The dashboard looked exactly like it always did, except the number in the corner was three weeks old and I’d been treating it as current.
That’s the specific danger with automated reporting: when it breaks, it usually doesn’t look broken. It looks like nothing happened. And “nothing happened” is very easy to read as “everything’s fine.”
This is common for anyone running a one-person business on a stack of scheduled scripts and dashboards, whether that’s a cron job pulling Stripe data, a Zapier flow updating a spreadsheet, or a BI tool refreshing a chart every morning. The point of automation is that you stop watching it. That’s also exactly why a quiet failure can sit there for weeks.
Why “no error” isn’t the same as “working”
A report can fail in two very different ways. A loud failure throws an error, sends you an alert, or shows an obvious blank space where a chart should be. Those are annoying but easy to catch, because something visibly changed.
A quiet failure is different. The job runs. It doesn’t crash. It just doesn’t do what you think it’s doing. A few common versions:
The script runs but the query returns nothing new. Maybe an API token expired, so every request gets rejected, but the script’s error handling logs the failure to a file nobody reads and then exits cleanly instead of crashing loudly.
The data source changed shape underneath you. A field got renamed upstream, a column moved, an integration got deprecated. The script still runs, but it’s now pulling an empty column or defaulting to zero instead of failing outright.
The report shows cached or stale data instead of a fresh pull. Some dashboard tools fall back to the last successful result when a refresh fails, specifically so the UI doesn’t look broken. That’s convenient for the tool and a trap for you, because the chart looks alive when it’s actually frozen.
A timezone or date boundary shift quietly duplicates or drops a day. This one is sneaky because the numbers still move, just not accurately, so nothing about the shape of the chart tells you something’s wrong.
A try/except block swallows the real error. This is the one I see most often in my own scripts. Somewhere early on, to stop one bad row from crashing an entire nightly job, I wrapped a step in a try/except that logs and continues. That’s reasonable for one bad row. It’s a problem when the whole data source starts failing and the same “log and continue” logic just quietly skips the entire run, day after day, and nothing ever escalates.
None of these are exotic. They’re the ordinary failure modes of any system that talks to another system on a schedule. The reason they matter more for reporting than for, say, a broken checkout page, is that a broken checkout page announces itself through lost revenue and angry customers. A broken report just sits there looking normal while you make decisions off it.
What actually gets missed when this happens
The real cost isn’t the outage itself. It’s the decisions made during the outage, and how long it takes you to notice.
Say your MRR report silently stopped updating for two weeks and you’re a one-person shop watching it as your main signal for whether to keep spending on ads. If the number was already high when the job died, you might keep spending under the assumption things are holding steady, when the truth is you have no current information at all. It’s not that the report lied to you. It’s that it stopped telling you anything, and you didn’t notice the silence.
Same story with a churn dashboard. If the count of cancellations stops updating, the chart doesn’t necessarily go to zero. Depending on how it’s built, it might just flatline at whatever the last real value was, which reads as “churn is stable” instead of “churn is unknown.” Those are very different situations to be making decisions in, and the chart won’t tell you which one you’re in.
I want to be clear about what I’m not saying here. A stale report isn’t a moral failing, and this isn’t an argument for checking every number by hand every day. Automation exists so you don’t have to do that. The point is narrower: the report itself can’t tell you it’s broken. You have to build something else that watches the report.
Building in a heartbeat, not just a chart
The fix that’s worked for me isn’t more elaborate error handling in the report itself. It’s a separate, dumb, boring check that runs alongside it and answers one question: did this job actually complete, with fresh data, in the last expected window?
A few ways to do that without much infrastructure:
Log a timestamp on every successful run, and check the age of that timestamp separately. If a job is supposed to run daily and the “last successful run” timestamp is more than 36 hours old, that’s worth a look. This is the single cheapest thing you can add to any scheduled script, and it catches most silent failures because it doesn’t care what happened inside the job. It only cares that the job finished.
Check row counts, not just values. If yesterday’s pull for a given source usually returns somewhere in the hundreds of rows and today it returns zero, that’s a signal even before you know whether the number is right. A sudden drop to zero rows, or an identical row count two days running, is often the tell that something upstream changed shape.
Alert on the absence of an event, not just the presence of an error. Most alerting is built to fire when something goes wrong loudly. A “no news” check flips that: if the job hasn’t reported success by a certain time each day, send yourself a message. This only takes a scheduled check that looks for a heartbeat file or a database row updated within the last day.
Spot-check against the source occasionally. Once a month or so, pull the raw number directly from wherever the data originates, whether that’s your payment processor’s own dashboard or a direct database query, and compare it to what your automated report shows. If they’ve drifted apart, you’ve found a problem before it compounded for another month.
None of this is complicated engineering. It’s the reporting equivalent of a smoke detector: cheap, boring, and only useful because you never think about it until the one day it matters.
What this doesn’t fix
A heartbeat check tells you a job ran. It doesn’t tell you the job ran correctly, and it definitely doesn’t replace understanding what the report is measuring in the first place. You can have a report that updates perfectly on schedule and is still tracking the wrong definition of churn, or double-counting a segment, or missing a channel entirely. Freshness and correctness are two separate problems, and solving one doesn’t touch the other.
It also won’t tell you when to trust a number less because of a small sample size, a seasonal blip, or a one-off event that skews a metric for a week. That’s a judgment call, and no monitoring script makes it for you.
What a simple freshness check does buy you is the confidence that when you look at a number, you’re looking at something current, not something that quietly stopped moving three weeks ago while you kept making calls based on it.
If you want more on how to think about the metrics and methods behind small-business reporting, and where the automation around them tends to quietly go wrong, you can find the rest of what I’ve written over at the homepage.
Get new guides and videos first — join the Telegram channel.