Building· 4 min read
Our personality test was calling agreeable people "fearful"
A quiz that gave the harshest of its four results to anyone who tends to agree with statements. It never threw an error — we found it by building a shorter version.
Our test puts people into one of four attachment patterns. One of them, fearful, is the heaviest thing it can tell you: you want closeness and you flinch at it. It's the result you'd least want to receive if it weren't true.
For a while, anyone who tends to agree with statements got it. Not because of anything about them. Because of how we'd counted.
The mechanism
Attachment gets measured on two scales — anxiety and avoidance — each built from several agree/disagree statements. Some are worded so that agreeing means more of the trait ("I worry they'll lose interest"), and some are reversed, so agreeing means less ("I'm confident they'll stick around"). Reversed items exist for exactly one reason: to catch people who answer the shape of the question rather than its content.
That mechanism only works if the two kinds are balanced. Ours weren't. Anxiety was four forward and two reversed. Avoidance the same.
Work through what that does to someone who picks "agree" on everything. The four forward items push their score up. The two reversed items push it down. Four beats two, so they land at about 67 out of 100. We call anything above 50 elevated. They're elevated on anxiety.
The identical arithmetic runs on avoidance, because it had the identical imbalance. Elevated on both. Elevated on both is fearful.
So the test would hand its most loaded result to someone whose only distinguishing feature was a mild tendency to say yes. Then it would print three paragraphs about their fear of intimacy, and match them with a companion chosen to accommodate it.
Why nothing caught it
There was no error. There was never going to be one. Every function returned a number in range, the percentages summed to 100, the page rendered, the report read beautifully. Acquiescence bias is a known problem with a standard fix and we had implemented the fix — we'd just implemented it at the wrong ratio, which looks identical to implementing it correctly right up until you compute the consequences.
Our test suite had an assertion for this. It said: every scale must contain at least one reversed item. All four scales passed. The assertion was checking that we'd had the idea, not that the idea worked.
How it surfaced
Not by review. We were building a 13-question version of the test — five minutes is a long time to ask of a stranger — and for a short version we picked one forward and one reversed item per scale, because with two items you don't have another option.
Then we wrote the obvious check: the short test and the full test should agree about the same person. They didn't. Answer everything "agree" and the short test said secure, the full test said fearful.
Our first instinct was that the short version was too crude. It wasn't. With one forward and one reversed item, agreeing with both cancels exactly, and you land on 50 — dead centre, no claim made. That's the correct answer to a content-free response pattern. The accidentally-balanced version was right and the carefully-designed one was wrong.
The fix
Rewording three statements into their reversed form, so all four scales sit at 1:1. "If plans change at the last minute, I tend to assume it's about me" became "When plans change at the last minute, I take it at face value." Same construct, opposite polarity.
And replacing the assertion with one that tests the behaviour rather than the machinery: answer every question the same way, and all four scales must land exactly on 50, and the result must be secure. Five variants, one per answer option. That assertion fails loudly on any future imbalance, including one introduced by someone adding a single well-meaning question.
What it changes for you
If you took our test before August 2026 and got fearful, it's worth retaking. It may well still say fearful — plenty of people are — but it should now be saying it because of your answers rather than because of our arithmetic.
The broader version of this: any test that sorts you into a named type is doing arithmetic on a scale someone built, and both the scale and the boundary are judgement calls. Ours cuts at the midpoint, which is a defensible default and not an empirical threshold. A result of 51 and a result of 49 get different names and are not different people. (The five love languages have a worse version of this problem, and we use those too.)
Which is why our short version now tells you when you've landed close enough to a boundary that one different answer would have flipped the label. It's less satisfying than a confident verdict. It's also true.