For months my paid conversion rate had a robot in it.
The phone rental side of what I run has a canary. A script that fires at seven minutes past nine every morning, signs up, pays, gets a device assigned, checks the device answers, then tidies up after itself. It exists so I hear about a broken checkout before a customer does, and it has caught two real breakages.
It is also a purchase. Every counter downstream of it thinks so.
One guaranteed sale a day works out around thirty a month. It lands in signups, in paid conversions, and in the funnel chart sitting between the two. I wrote the script. I know its schedule. I read that conversion figure for months anyway without once subtracting it.
What gave it away in the end was how calm the number was. Small numbers jump around. Mine had a floor it never quite fell through, because a machine was holding it up every morning.
The figure was never wrong. I just never asked what was inside it, and the dashboard had no way to volunteer.
Nobody is deceiving you
Most writing about misleading charts is about somebody misleading somebody else. A vendor’s slide, a government press release, a startup’s fundraising deck.
The dashboard you built for yourself is a different animal, and a harder one, because there is no adversary to be sceptical of. The honest chart and the flattering chart cost the same to build. The flattering one usually wins because it is what the tool draws before you touch anything.
So the question to hold in your head is which decisions this chart made on my behalf while I was busy building it.
There are four of those decisions, and every one of them changes what the picture says without touching the data.
The four decisions the chart makes for you
The first is the axis. A charting tool picks a vertical range that makes the line look interesting, which almost never means starting at zero. It zooms until the wiggle fills the frame.
I had a chart of data used per line where the vertical axis started at one hundred eighty gigabytes. A drift from one hundred ninety to two hundred and five looked like a wall going up. In reality it was eight percent, well inside what a few customers changing their habits do to you in a month.
Cropping an axis is not automatically dishonest, which is what makes it slippery. If you are watching uptime and uptime lives between ninety eight and one hundred percent, a chart starting at zero is a flat line that tells you nothing. The rule is to know which sort you are looking at in the half second before you form an opinion, and the cheapest way to guarantee that is to put the axis floor in the title. Ugly, and it takes the drama out immediately.
Bars are the exception with no defence. A bar states its meaning through its length, so cropping the axis under a bar chart draws one bar three times the height of another to show a six percent gap. That is just a wrong picture.
The second decision is smoothing. A seven day rolling average is the most common statistic on any small business dashboard and it usually earns its place, because raw daily lines are unreadable once weekends are in them.
Then you get a day like the one in July where a provisioning bug meant any port name shorter than four characters failed silently. New lines came up dead for most of a day. On the raw daily chart that is a spike to the floor you cannot miss. On the seven day average it is a shallow sag that I would have scrolled straight past, because the averaging took the most important event of the month and spread it thinly across a week until it looked like weather.
Sometimes the volatility is noise you want removed. Sometimes the volatility was the finding, and you have paid a filter to delete it. Draw the raw line faintly under the smoothed one and you never have to guess which case you are in.
The same trap wears a second costume: aggregation. A modem on one of my servers used to drop its connection for a stretch in the small hours and recover on its own. The daily chart showed a good day. The hourly view showed a hole every night in the same place. Nothing was smoothed. The bucket was just wider than the problem.
The third decision is colour. Every tool ships with up is green, down is red. On revenue that is correct. On churn, refunds, and support ticket volume it is exactly backwards, and the tool has no idea which of those it is drawing. I had a sparkline sitting cheerfully green for weeks. The number climbing was cancellations.
Colour is faster than reading, which is what makes it dangerous. By the time you have read the label you already have a feeling, and that feeling came from whoever picked the palette. Somebody decided which direction counts as good, and the chart is now stating that decision as though it were a property of the data.
The fourth is the second axis. A dual axis chart lets you choose two scales, and choosing the scales chooses where the lines cross. I can put ad spend and signups on one, slide the right hand axis, and produce a picture where spend clearly leads signups. Slide it the other way and signups lead spend. The data does not move at all.
A rate with no count is a claim it cannot support
This is the one that costs small operators real money, and it is separate from the four above because it is about what the chart leaves out rather than how it draws.
A tile says activation rate, one hundred percent. Wonderful. It is three people out of three.
At small volume a percentage is a staircase. It can only land on a handful of values, and one customer moves it by twenty five or thirty three points. The chart draws a smooth line sloping between those values, which is a picture of something that never happened.
Print the count beside the percentage. Everywhere, including the little summary tiles. One hundred percent, three of three. It reads worse and it stops you dead.
The related version is the silent unit switch. One tile shows refunds as a count, the tile next to it shows refund rate, same size and same colour and same row, and your eye files them as the same kind of thing. Put the unit on the face of the tile rather than in a tooltip. Nobody hovers.
Say the number before the page loads
Here is the habit that has done more for me than any charting rule.
Before the dashboard renders, say what you expect to see. Signups this week, about eleven. Cancellations, one or two.
Then look. If you are close, you understand your own business and the check took ten seconds, so go and do something else.
If you are miles off, you are in one of two worlds. Either the business changed, or the measurement changed. The measurement is the more common answer and almost nobody checks it first.
A tracking tag that stopped firing on one page after a deploy. A webhook that retried and wrote every event twice. A timezone setting that quietly moved a day of activity into the day before. A crawler working through your checkout at four in the morning.
Treat a surprise as a bug report until proven otherwise. Open the raw table, count some rows by hand, check the number at its source. Five minutes. The alternative is building a theory about customer behaviour on top of a broken counter, and theories are expensive to take apart later.
A metric with a target on it stops measuring
A number stops measuring what it used to measure the moment somebody’s performance is judged on it. People quote that at managers. It happens to you on your own, in a room, with nobody watching.
The subtle version is definition drift. Set yourself an uptime target and the cheapest way to hit it is to quietly narrow what counts as down. A partial outage on one server, a customer who could still connect but at a third of the speed, a scheduled window you decided does not count. Nobody makes that decision dishonestly. You make four small reasonable calls over six months and the metric has drifted underneath you.
Keep the numbers you steer by away from the numbers you grade yourself on. And if a metric has a target sitting next to it, read it as a target, because that is what it now is.
Delete the charts nobody has acted on
Open your dashboard and write, beside every chart, the date it last changed something you did.
Most will not have a date.
I kept a country breakdown for a year and looked at it a couple of hundred times. I can name exactly zero decisions that came out of it. It was interesting. Interesting is not the bar.
The test I use now is whether I can say in one sentence what I would do if that chart moved. If the sentence does not arrive, the chart goes. Delete rather than hide, because hidden charts come back.
Deleting is harder than it should be, since building the chart was work and it took an afternoon and it looks good. Sunk cost with a colour scheme on top. But a crowded dashboard is a way of being busy with your business without deciding anything about it, and you can keep that up for years. It looks like diligence from the outside, and worse, it looks like diligence from the inside.
The one I have not fixed
I still choose the flattering comparison. The same number against last week, last month and last year tells three different stories, and I reach for whichever one is kindest without deciding to. I notice afterwards, if at all.
The only defence I have found is fixing the comparison inside the chart so there is nothing left to shop for, and I still catch myself opening a second view when the first one is unkind.
None of this makes a number true either. A clean chart of a badly defined metric is still wrong. It is just legible now, which if anything makes it easier to believe. The rest of how I read my own numbers is written up on the home page.
Get new guides and videos first — join the Telegram channel.