You send a survey to your customer list. Two hundred people get it, thirty reply. You look at the replies and something feels off: almost all of them are your longest-tenured customers, the ones who’ve been with you two years or more. Your newer customers, the ones who joined in the last three months, barely show up. But new customers are half your actual customer base.
So now what do the survey results mean? If 80% of respondents say they love your onboarding, is that true of your customers, or just true of the loyal core who was going to answer a survey anyway?
This is the exact problem survey weighting is built to handle. It’s also a problem that gets genuinely tricky once your sample is small, which for most one-person businesses, it is.
What weighting actually does
Weighting doesn’t add data you don’t have. It doesn’t guess what the non-responders think. All it does is change how much each response counts, so that the mix of respondents matches the mix of your actual customer base.
Say your customers break down like this: 50% joined in the last three months, 50% have been around longer. But your survey responses break down 20% new customers, 80% long-tenured. Weighting says: since new customers are underrepresented by a factor of 2.5 in your responses (20% actual vs 50% needed), count each of their responses as worth 2.5 votes instead of 1. Long-tenured respondents are overrepresented, so their votes get scaled down.
The formula behind this, called post-stratification weighting, is simple: for each group, divide the group’s true share of your population by its share of your sample. That ratio is the weight applied to every response in that group.
New customer weight = 50% (true share) / 20% (sample share) = 2.5
Long-tenured weight = 50% (true share) / 80% (sample share) = 0.625
Multiply each respondent’s answers by their group’s weight before you average anything. The math is not the hard part. Knowing whether you should trust the result is.
A worked example with plan tiers
Let’s walk through a case with round numbers, purely to show the mechanics, not as a real result from anywhere.
You run a subscription tool with three plan tiers: Basic, Pro, and Team. Your actual customer base is 60% Basic, 30% Pro, 10% Team. You send an NPS survey and get 50 responses back: 20 from Basic, 20 from Pro, 10 from Team. So your sample is 40% Basic, 40% Pro, 20% Team, quite different from your real mix.
Weights come out to:
Basic: 60/40 = 1.5
Pro: 30/40 = 0.75
Team: 10/20 = 0.5
If your Basic respondents scored an average NPS of 40, Pro scored 20, and Team scored 10, the unweighted average across all 50 responses is a simple mean of the raw scores. The weighted average instead multiplies each tier’s average score by its weight and its response count, then divides by the total weight. Because Basic is underrepresented in your raw sample relative to its size in your customer base, weighting pulls the overall number toward Basic’s score, which is closer to what your full customer base probably feels, assuming Basic customers who didn’t answer feel similarly to the ones who did.
That last clause is the part people skip past, and it’s the whole ballgame.
Why small samples break the math
Weighting corrects for one thing only: the proportion of each group in your sample versus your population. It cannot correct for the people within a group who chose not to answer at all, and it cannot manufacture precision that isn’t there.
Two things go wrong specifically at small sample sizes.
First, cell counts get thin. If your Team tier only pulled in 4 responses instead of 10, and Team makes up 10% of your customer base, the weight on those 4 people gets very large. Now 4 people are effectively speaking for a much bigger slice of your reported result. One outlier opinion in that group of 4, someone who had an unusually bad or good week, can swing your entire weighted number more than it should.
Second, weighting doesn’t grow your effective sample size, it can shrink it. When weights are very uneven, the statistical “effective sample size” of your survey drops below the raw count of respondents. Thirty raw responses with wild weights might behave, in terms of reliability, more like eight or ten responses. Nothing in the math tells you this loudly. The weighted average just comes out looking like a normal, precise-seeming number, and it’s easy to report it with more confidence than it deserves.
This is the trap: weighting makes a skewed sample look like it represents your customers, but the version of “representing” it delivers is about proportions, not about certainty. A weighted result from 30 responses is still a result from 30 responses.
Guardrails if you weight anyway
If you’re going to weight a small survey, a few habits keep it honest.
Keep the number of weighting variables to one, maybe two. Weighting on plan tier alone is manageable. Weighting on plan tier and signup cohort and geography at the same time, with a 40-person sample, creates dozens of tiny cells and weights that stop meaning anything.
Set a floor on cell size. If a group has fewer than 5 or so responses, don’t weight it on its own, either fold it into a neighboring group (combine Team with Pro, for instance) or report it separately and flag it as too thin to weight.
Cap extreme weights. A common practical rule is to not let any single weight go much above 3 or 4x, even if the raw math says higher. If your formula spits out a weight of 8 for a group of 3 people, you have stopped correcting anything and started letting three people dictate the story.
Show both numbers when you report the result internally. If unweighted NPS is 25 and weighted NPS is 38, that 13-point gap is telling you something about who responded, and it’s worth sitting with rather than just publishing the number that looks nicer.
When to skip weighting and just say so
For a lot of one-person operations, the more honest move is to not weight the sample and instead just report who it came from. “Of the 30 people who responded, 80% have been customers over a year. Here’s what they said, and here’s why I’m not treating this as representative of newer customers.” That sentence does more for decision quality than a weighted average that quietly implies precision the data doesn’t have.
Weighting earns its keep when you have a reasonable number of responses in every group you’re correcting for, generally people who work with survey data look for dozens per cell, not single digits, before trusting a weighted figure much. Below that, weighting is still a legitimate technique, but it’s adjusting proportions on a foundation that’s already thin, and it’s worth treating the output as directional rather than a solid number to build a decision on.
The bottom line
Survey weighting is a proportion fix, not a magic trick. It makes the mix of who answered match the mix of who you actually serve, and that’s genuinely useful when your respondents skew toward one type of customer. What it can’t do is tell you what the people who never opened the survey think, and it can’t turn a handful of responses in a thin category into a reliable number just because you multiplied it by a weight.
If you’re running a survey with a small list, decide up front how many responses you need in your smallest important group before you trust a weighted result, and be ready to just describe the skew in plain terms if you don’t get there. That’s not a failure of the survey. It’s the survey telling you the truth about its own limits.
If you want more of this kind of plain-English walkthrough on the numbers behind running a small business, from churn and cohorts to A/B tests and data quality, you can find the rest of what we’ve covered on the home page.
Get new guides and videos first — join the Telegram channel.