How long should the answer be? Right-sizing LLM output

Ask a room of product teams what they've designed about their AI feature's output and you'll hear about tone, formatting, citations, streaming, guardrails. Ask how long the answers should be and the room goes quiet. Length is the one dimension almost everyone leaves to the model — and the model, bless it, answers every question like it's being graded on thoroughness. A two-word query about opening hours gets four paragraphs, a summary, and a cheerful offer to help further. The user wanted Tuesday.

Length is a product decision. It's set by the surface the answer renders on, the task the user is performing, and the cost of the words themselves. Teams that design it deliberately ship assistants that feel crisp and trustworthy. Teams that don't ship confident walls of text. Here's how we design it.

The model's default is wrong for almost every surface

Language models are trained to be complete and helpful, which in practice means long. Left alone, theyfront-load caveats, cover adjacent questions nobody asked, and close with a summary of the summary. In a benchmark environment that thoroughness scores well. In a product, where the answer competes with the user's actual task for screen and attention, it's noise.

The mismatch is sharpest on constrained surfaces. We've shipped AI features into a chat panel, an inline editor, a notification, and an email digest on the same client product. The same underlying answer had four correct lengths — one sentence for the notification, a short paragraph inline, a structured few paragraphs in chat, and a scannable digest section in email. A single "answer the question" prompt produced one length that was wrong four different ways.

So the first move is boring and powerful: give every surface an answer budget. Not a vibe — a number. Ninety characters for the toast. Sixty words inline. Two hundred words for chat, more only when the user asks to go deeper. Budgets force the design conversation early: what does the user need here?

Summary–detail layering beats one right length

Most disputes about answer length dissolve when you stop trying to pick one. The pattern that keeps winning our critiques is layered disclosure: a short answer that stands alone, with explicit affordances to go deeper.

Concretely:

• Lead with the answer. First sentence carries the decision, the number, the yes-or-no. Not the reasoning, not the context. "Yes — your plan covers this" beats three sentences of throat-clearing about how plan coverage works. This is just good writing, but models default to essays, so it must be specified, prompted, and enforced in the rendering layer.
• Structure the detail as expansion, not continuation. "Why", "Show working", "Source", "More options" — small controls that let the user pull more instead of having more pushed at them. Each expansion is its own designed moment, with its own budget.
• Let verbosity be a user setting where it genuinely varies. Some contexts split down the middle: domain experts want terseness, newcomers want scaffolding. A per-user or per-conversation dial ("shorter" / "more detail") that persists beats re-prompting the model every turn. Fold it into the kind of preference memory we describe in assistant memory design — state it, store it, honour it everywhere.

Layering also plays beautifully with streaming: stream the short answer first, render follow-on depth behind controls. The user reads the headline while the rest arrives.

Designing the budget into the system

A budget written in a strategy deck is a wish. It becomes real in three places:

The prompt layer. Length instructions in the system prompt work, but softly. "Answer in under 60 words" moves the median; it doesn't bound the tail. Treat prompt-level instruction as the first draft of the constraint, not the enforcement.

The structure layer. This is where structured outputs shine. If the answer is a JSON object with a headline field (max 10 words) and a detail field (max 60), the budget is now a schema, and schemas are enforceable, validatable, and testable. We increasingly render LLM answers from structure rather than from prose precisely because structure lets us design the length instead of hoping for it.

The rendering layer. The UI is the last line of defence: clamp, collapse, or paginate over-long output with a visible "show more", and log when it triggers. A renderer that never clamps trains the team to tolerate unbounded answers; a clamp that fires constantly is a measurable signal that the model isn't holding its budget. Either way, the constraint is real and observable.

Too long and too short are both failure modes

Here's the part most evaluation suites miss entirely. Teams measure accuracy, hallucination, latency, toxicity — and never once score length fitness. Yet "rambled" and "wasn't enough" are among the most common user complaints about AI features in every round of testing we run.

Treat both as first-class failure modes in your evals:

• Over-length evals. For a golden set of simple questions ("what time do you close?"), flag any answer exceeding its surface budget. You'll be shocked how often it fires on day one, and how fixable it is once visible.
• Under-length evals. For complex questions, check the answer carried the required substance — did it include the deadline, the caveat that changes the decision, the number? Terseness that omits the load-bearing fact is worse than verbosity, because it looks confident while failing.
• Proportion evals. Answers should scale with question complexity. A rubric that scores "answer effort proportional to question effort" catches the comedy cases: five paragraphs for a yes/no question, one shrug of a sentence for "compare these three plans".

And measure the economics alongside the experience. Output tokens are the expensive half of the bill, so every unnecessary paragraph is a tax you charge yourself — the arithmetic in cost engineering for LLM features applies sentence by sentence. A disciplined answer budget routinely cuts output cost per session by a third while improving satisfaction. Length discipline is one of the rare places where cheaper and better are the same change.

A note on the exceptions

Two contexts where long answers earn their pixels. First, teaching surfaces: when the product's job is to build the user's understanding (an explainer, a tutor, documentation assistance), the detail is the value and the budgeting flips — ration the repetition, not the explanation. Second, deliberative tasks: compare-and-decide over real stakes. There, comprehensiveness is the ask, and the design move is structure — tables, sections, cited sources — not truncation. Even then: lead with the verdict.

Key takeaways

• Answer length is a product decision set by surface and task. Leaving it to the model means accepting a default that was tuned for benchmarks, not users.
• Give every surface an explicit answer budget — characters or words — and let the budget drive prompt, schema, and renderer alike.
• Prefer summary–detail layering over finding one perfect length: lead with the answer, expose depth behind deliberate controls.
• Enforce budgets in structured output schemas and in the rendering layer; prompts alone set medians, not bounds.
• Score over-length and under-length as first-class eval failures, with proportion to question complexity as the rubric.
• Shorter, budgeted answers are usually cheaper and better — one of the few times the economics and the experience agree.

FAQ

Won't strict budgets make the assistant feel clipped or robotic?

Only if the short answer is badly written. A single well-crafted sentence with the verdict and the one fact that matters feels more premium than three paragraphs of hedging — the same way conversion copywriting teaches that clarity reads as confidence. Brevity done with craft is warmth; brevity done by truncation is rudeness. The budget constrains quantity, not quality.

Should the user be able to ask for more after a short answer?

That's the design. "Tell me more", "why?", and expansion controls are the user's side of the budget — depth on demand. The failure mode to avoid is forcing the user to ask for less, repeatedly, every single turn.

How do we pick the numbers for each surface's budget?

Prototype with real content at 375px width and count what fits without scrolling for a glanceable task — that's your ceiling. Then halve it for the target. Validate in usability testing by watching where eyes leave the answer; the budget is right when users read the whole thing and act.

Do budgets change across models or model versions?

Model behaviour drifts — one version's "concise" is the next version's paragraph — which is why budgets belong in schemas and renderers, not only in prompts, and why length evals belong in the migration checklist whenever you swap models. The surfaces don't change; the enforcement shouldn't either.