Choosing an agency: signals over showreels

I've sat on both sides of the agency selection table — eleven years pitching Brassfern, and, in a previous life, two years inside a company buying exactly this kind of work. Here's the uncomfortable summary from the buying side: the selection process most companies run is optimised to find the best pitcher, not the best builder. Showreels reward studios that ship beautiful launches (all of them can show highlights; nobody shows the maintenance). Chemistry calls reward charisma (sales directors have more of it than engineers). Scoring matrices reward compliance (ticking boxes is a proposal skill, not a delivery skill).

You can invert all three. This is how, and it costs you nothing but slightly different questions.

The showreel tells you one thing — ask for the boring evidence

A portfolio proves a ceiling: at their best, with a client that let them, this studio produced this. Useful, insufficient. What predicts your project is their mean, and you get at the mean with process evidence:

Ask to see a shipped thing that's two years old, still live, and browse it on a mid-range phone over hotel wifi. A studio's real quality bar is what's survived two years of content updates, marketing scripts and organisational drift. (We subject ourselves to this: poke around the Fernleigh Wines storefront and the Northwind Ledger dashboard on the worst device you own.)
Ask for the artefact trail from one past project: the brief they received, a mid-project status report, a change request, the handover document. You're not auditing them; you're checking that the trail exists. Studios with genuine process produce these in an afternoon. Studios cosplaying process produce a pause.
Ask what they measure. Not "do you care about performance" — everyone cares. "What was INP on your last three launches at p75" has an answer or it doesn't. The gap between a studio that reports in numbers and one that reports in adjectives is the entire engagement, previewed.

Who actually does the work

The most reliable predictor of project quality is the seniority of the people in your weekly call. The most reliable way to be misled about it is the proposal's team page, which can legally contain the founders.

Fix it contractually, not conversationally. Require named individuals with roles and percentage allocations in the proposal, and a substitution clause: named key people can't be swapped without your consent and a rate adjustment. Then ask one disarming question in the pitch: "Which of these people will personally be in our Tuesday call, and what will they be doing the previous Tuesday?" The answer — or the tap-dance — tells you whether you're buying the team in the room or the team you haven't met.

Related test: ask who their best client would describe as the most demanding person at the studio, and then ask to meet that person. In every good studio there's one — the person who says no to shipping the mediocre version. If the studios can't produce them, either they don't exist (bad) or they're being hidden from clients (worse, and telling).

Reference calls that reveal truth

Standard references are theatre: the studio selects its happiest client, the client is polite, everyone learns nothing. Two adjustments make references investigative:

Take references on projects that went sideways. Ask: "Can I speak to a client where the project hit real trouble — a missed date, a scope fight, a launch incident?" Every honest studio that has existed for a decade has these. The ones that decline because they have no troubled projects are lying; the ones that decline because they won't show you are the finding. What you want to hear from that reference isn't that nothing went wrong — it's who called whom first, how the bill was handled, and whether the relationship survived.

Ask reference questions with a correct answer:

"What did they push back on, and were they right?" (A studio that never pushed back never protected you.)
"What did the invoice look like versus the quote, month by month?"
"Who left the project mid-way?" (Everybody has churn; you want to know how it was handled.)
"Would you hire them for a different kind of work — and what kind of work would you not give them?" A studio with no honest boundary is a studio that says yes for a living.

Pricing structure as a personality test

Ignore the number for a moment and read the structure; it's an X-ray of how the studio manages risk and honesty. Red flags we've seen from the buying side:

A single fixed price with a vague scope. Fixed price is fine; fixed price without a scope you could test in court means the risk lives in a change-order machine, and you'll meet it in month two. (The honest version of fixed pricing exists — we describe ours on the pricing page — but the scope has to be real.)
Rate cards with no team. $150/hr means nothing until you know whether the hours are the principal's or the intern's.
Discovery "credited back" if you proceed. This prices the discovery at zero, which guarantees it will be worth zero — a sales step dressed as diligence.
Payment schedules weighted to signature (say, 60% on signing). Studios that front-load cash are financing operations with your anxiety. Healthy schedules track delivery.

And the inverse flag — the good one: a proposal that shows its assumptions, lists what's excluded, and prices a smaller phase one with an explicit decision gate. That structure costs the studio something at proposal time; that's why it's trustworthy. For the full anatomy of a quote, read how studios price and how to read a quote.

Design the chemistry call so chemistry can't carry it

The chemistry call's job is to answer one question: can this team think about my problem out loud? Which means you must let them. The format that works: send the brief a week ahead (a proper one — our brief playbook is the ten-minute version), then run a 60-minute call where they spend 40 minutes interrogating your problem and proposing directions, and you spend 20 minutes redirecting them with a deliberately changed constraint: "Assume the budget halves," or "assume the date moves up eight weeks."

You're watching for three things: whether they ask evidence-seeking questions or opinion-seeking ones; whether changing the constraint changes their answer (a studio with one solution keeps recommending it at every budget); and whether the people thinking out loud are the people named in the proposal. Charm is lovely. Charm under a changed constraint is data.

The decision itself

When the proposals are in, the temptation is a weighted matrix. Use one, but weight it the way projects actually fail: team fit and evidence of thinking over price and polish. Our suggested weighting from the buyer's seat:

Named team & seniority: 25% — the single strongest predictor.
Response to your specific problem: 30% — did they engage with your evidence, or template it?
Process evidence & references: 25% — the mean, not the ceiling.
Price & structure: 20% — and score structure (assumptions, exclusions, gates) as heavily as the number.

Then do one last thing before signing: call the studio you've chosen and tell them the second choice, asking "what would you have asked them that I might not?" You'll get a candid self-assessment worth more than the matrix, and you set the relationship's tone: you bought honesty, and you intend to keep receiving it.

Key takeaways

Portfolios show ceilings. Ask for two-year-old live work, artefact trails and measured numbers to find the mean.
Contract the team, not the brand: named individuals with allocations and a substitution clause, verified in the pitch.
Reference-check the troubled projects and ask questions with correct answers: pushback, invoice vs quote, mid-project exits, boundaries.
Read pricing structure as personality: vague scope plus fixed price is a change-order machine; assumptions and exclusions are the trust signal.
In the chemistry call, let them interrogate your problem, then change a constraint mid-call and watch the thinking.
Weight the decision toward team and thinking — 25/30/25/20 — and price structure, not just price.

FAQ

Should we run a paid trial project before committing?

Yes for large engagements, with one rule: the trial must be real work with real deliverables and a real decision gate — a paid discovery sprint, a spike, a designed-and-built single page — not a speculative design exercise. You learn how they work; they learn whether you're a good client. Both datasets matter. Our own discovery sprint format exists precisely to be that trial.

How many agencies should we invite to pitch?

Three, plus a pre-qualified reserve. Two invites tells the winner there's no real competition; seven burns everyone's best people and selects for whoever has the largest pitch team. Respect the process and the quality of what you receive goes up — agencies spend their best hours on winnable, well-run pitches.

Is bigger safer?

Bigger is different: more redundancy, more process, more distance between you and the senior people. For a platform your company lives on, redundancy matters. For a marketing site or a product sprint, a senior team of five will usually outrun a department of forty. Match the agency's shape to the failure mode you're actually afraid of.

What if the best proposal came from people not in the pitch call?

Treat it as a finding, not a dealbreaker: ask to meet the actual delivery team before signing, run one working session with them, and make the named-team clause non-negotiable. Some studios pitch with leaders and deliver with a strong but different team; that's acceptable only if you meet and choose that team with eyes open.

When should we walk away from the whole process?

When every proposal is solving a different problem — that's a signal the brief failed, and no selection will fix it. Go back, do the brief work (evidence, constraints, budget band), and re-run. Two weeks of re-briefing costs less than selecting the best answer to the wrong question. If you want a second set of eyes on the brief itself before it goes out, the contact page reaches a human who reads briefs for a living — ours included.