Testing brand voice on people, not vibes

Every voice project reaches the same meeting. Two directions on the wall — say, one dry and precise, one warm and chatty — and the room decides by vibe. The senior person picks a favourite, someone references a brand they admire, the junior designer who preferred the other one goes quiet, and a company-wide voice is chosen the way you'd pick a restaurant. Sometimes it works. Sometimes you spend two years sounding like a chatbot that did improv once.

The strange thing is that we test almost nothing else this way. Headlines get A/B tested, pricing pages get research rounds, checkout flows get instrumented to the click. But the voice — the thing customers read — gets installed by executive preference and defended by taste. This piece is about fixing that: five tests we run with actual humans before, during and after a voice engagement, cheap enough to fit in a normal project budget and structured enough that the results survive a sceptical boardroom.

Why the vibe check fails

Taste isn't the problem — experienced people have good taste. The problem is that the room answers the wrong question. Asked "which do you like?", stakeholders hear "which flatters your self-image?". A CFO reads the witty option and imagines the company she wishes she worked at. Nobody in the room is a first-time customer on a phone at 11pm trying to figure out whether their card was charged.

There's a second failure: the room reads voice samples under laboratory conditions, with full attention, twice. Real readers give your copy a fraction of a second of partial attention. A voice that delights under scrutiny can exhaust at speed, and vice versa — dry copy that seemed cold in the workshop turns out to be the fastest read in the wild. You can't feel your way to that answer from a boardroom chair.

Test 1: comprehension first-run

Before any question of charm: does the copy work? We recruit five to eight people loosely matching the actual audience — customers in market, not colleagues' partners — and give them a key piece of service copy: the pricing explainer, the claims process, the "how billing works" page. They read it once, then we take it away and ask them to explain back what happens, what it costs them, and what they'd do next.

The output isn't opinion, it's error. "They thought the free trial auto-converted when it doesn't." "Nobody could say when the refund arrives." Comprehension failures are voice failures in disguise: unclear copy is usually unclear because it was written to sound a certain way instead of to be understood, and this test makes that impossible to argue with. When we rebuilt the voice for Copperline Mutual, comprehension testing on the fee schedule was the evidence that ended two years of internal debate — nobody defends a sentence that half the panel couldn't paraphrase.

Test 2: the cloze deletion

Comprehension tests measure clarity, not texture. For texture — does the voice sound like this company — we borrow a language-learning exercise. Take published copy from the brand and blank out the adjectives and attitude-carrying words: "We're ___ about keeping your money safe." Generate the same cloze from two or three competitors. Hand participants the shuffled stack and ask them to fill in the blanks, then to sort the passages by company.

Two readings matter. If participants can't tell your copy from your competitor's copy, the voice is generic regardless of how it reads — a real and common finding. And if their chosen fill-ins skew sharply away from the intended register (the "confident" brand gets completed with "obsessed", "terrified", "here to disrupt"), the copy is projecting something you didn't plan. Both results survive boardroom scepticism because they're the audience's words, not the agency's adjectives.

Test 3: brand-vs-bland A/B reads

The most pragmatic test we run, and the one growth teams adopt fastest. Take one high-traffic piece of copy — homepage hero, onboarding invite, first lifecycle email — and write two versions: the current baseline, and the new voice applied properly (not a caricature — the committee-safe parody of the new voice is a favourite sabotage tactic, so generate the treatment with the people who wrote the guidelines). Then put it in front of humans with a task, not a preference question: read it, then answer a two-question quiz on content, then a short set of forced-choice attributes ("this company seems more… established or experimental? careful or fast?").

Preference questions ("which do you like?") re-invite the vibe check. Comprehension and attribute questions measure what the voice did. If the witty version reads just as fast and shifts "experimental" by thirty points without costing "trustworthy" anything, you have evidence, and the debate changes shape entirely. The statistics bar stays where it belongs: this is directional evidence from small samples, so report it as such — the honesty principles in our A/B testing statistics guide apply word for word. A directional result plus conviction beats a false-precision result with a p-value stapled on.

Test 4: "could a competitor say this?"

The cheapest instrument in the kit, and the one internal teams adopt most. Take any sentence about to ship and ask: could our three closest competitors publish this exact sentence without anyone blinking? If yes, the sentence is doing no identity work. "We're passionate about great service" — everyone can say it; it's furnishing, not voice. "We'll tell you when the cheap plan is the right plan, even when it costs us" — that's a stance, and stances are what voices are made of.

Run the filter as a standing agenda item in content review, especially on the highest-visibility copy. It pairs well with a written voice chart: the chart teaches the register, the filter enforces distinctiveness. One without the other produces either well-calibrated blandness or distinctive chaos.

Test 5: the longitudinal drift audit

Voice isn't tested once; it decays on a schedule. After launch, fold validation into the standing maintenance loop — if you run the monthly 15-minute voice test, extend it quarterly with a human panel: a handful of pieces published this month, run through a shortened cloze and attributes exercise with five fresh readers. The metric you're watching isn't quality, it's drift — the attribute profile sliding back toward "generic professional services company" as new writers inherit the keyboard.

This is also where you catch the slow substitutions that never trip a style check: the creeping increase in hedge words, the exclamation marks multiplying in lifecycle email, the support macros quietly reverting to tribunal. Drift is normal physics; the test just makes it legible before it compounds into a rebrand.

Folding it into a voice scorecard

All five tests feed one artefact: a one-page scorecard that any writer — or hiring manager, or agency partner — can apply to a piece of copy in about four minutes. Structure it as checks, not vibes:

• Clarity: could a first-time reader paraphrase the ask? (From test 1.)
• Register: place this on each published axis — where's it sit versus its target? (From the charts.)
• Distinctiveness: pass the competitor filter? Which specific words carry the identity? (From tests 2 and 4.)
• Consistency: does it match the three nearest published pieces in register and pronoun discipline? (From test 5.)
• The evidence column: link the last test that supports a contested ruling. A scorecard that can't cite its own evidence becomes taste with a spreadsheet over it.

The scorecard's real job is conflict de-escalation. "The voice research showed nobody could paraphrase the passive construction, and this headline failed the competitor filter" ends meetings in a way that "I don't think it's on brand" never will.

What this changes about selling voice work

An honest word about cost: the full panel work above adds roughly a week to a voice engagement and a few thousand in recruiting. What it buys is the thing voice work usually lacks — a trial. Voice guidelines unprotected by evidence are one reorg away from the bin. Voice guidelines that arrive with comprehension deltas, cloze results and a drift baseline get defended in rooms you're not in. For a discipline whose output is literally every word the company emits, that's the cheapest insurance in the entire brand budget.

Key takeaways

• Boardroom vibe checks pick the voice the room wishes it had, not the one readers process fastest. Test on the actual audience.
• Comprehension first-runs expose clarity failures that voice debates paper over.
• Cloze deletion tests whether the voice is distinguishable from competitors — often it isn't, and that's the finding.
• A/B read with comprehension and attribute questions, never "which do you like" — and report small-sample results as directional.
• The "could a competitor say this?" filter belongs in standing content review, not just the guidelines PDF.
• Fold everything into a scorecard with an evidence column, and re-test drift quarterly with fresh humans.

FAQ

How many people do we need for voice testing?

Five to eight per round for the qualitative tests (comprehension, cloze, attributes), fresh each round if you can — the tests are reading calibration, not prediction. Keep recruitment loose but honest: actual or plausible customers, screened only so they're not copywriters themselves. Small samples are fine because you're hunting errors and direction, not estimating a population parameter.

Won't testing flatten a brave voice toward the median?

Only if you test preferences. Preference questions sand off edges; comprehension and attribute questions don't — they tell you what the edges do. In practice, testing makes braver voices more likely to survive, because the bold option stops being "the agency's taste" and becomes "the option that shifted 'distinctive' forty points with no clarity cost".

Should we A/B test voice in production with real traffic?

For high-traffic, low-stakes copy — lifecycle email subject lines, invite flows — yes, with pre-registered metrics and kill criteria, treating it like any honest CRO experiment. Don't production-test anything regulatory, pricing-critical or complaint-prone; that's what the human panel is for. And never read a conversion delta on one email as a verdict on the entire voice — voices compound across surfaces.

How does this hold up if AI drafts most first passes?

Better than the manual era, honestly: you can now generate the cloze set, the competitor stack, and the treatment variations in an afternoon, so the human testing hours go further. The tests themselves must stay human, obviously — a model "evaluating" voice for distinctiveness tells you how the average of the internet feels about you, which is precisely the voice you're trying not to have.