Tag
#model routing
2 pieces filed under “model routing”.
AI · 14 May 2026 · 11 min
Latency budgets for LLM features
An LLM feature that answers correctly in eleven seconds is broken. How we set per-surface latency budgets, route models for speed, and kill what can't make the time.
LLM latencyperformance budgetsstreaming UX
AI · 24 Apr 2025 · 12 minCost engineering for LLM features
Unit economics for AI features: model routing, caching layers, context trimming, batch APIs, and dashboards that translate tokens into dollars per user.
LLM costsunit economicsmodel routing