#AI engineering
7 pieces filed under “AI engineering”.
Prompt injection: the threat model your LLM feature needs
A defensive playbook for shipped LLM features: untrusted-content labelling, tool privilege separation, output screening, canary tokens and red-team CI.
Structured outputs: making LLMs renderable
Free-text LLM output is a compatibility bug waiting to ship. JSON schema modes, constrained decoding, boundary validation, and safely rendering streamed partial objects.
Evals-first development: the build order that actually works
Write the golden set before the prompt, version prompts like code, and gate merges on eval scores. The workflow that makes LLM features shippable.
Shipping LLM features: a production checklist
The pre-launch checklist we run before any LLM feature ships: evals, traces, cost ceilings, fallbacks, feedback loops and the metrics that catch quality drift.
Evals are the new unit tests: a practical guide for product teams
Golden sets, rubric graders, regression gates and CI plumbing: how product teams build LLM evaluations that catch real regressions without a research team.
Evals before features: how we measure LLM quality
How we build evaluation harnesses for LLM features: golden datasets, rubric graders, human review lanes and regression tracking that stakeholders can read.
RAG pitfalls we hit so you don't have to
Retrieval in production humbles everyone. The chunking trade-offs, citation bugs, stale knowledge and eval failures that taught us how RAG actually breaks.