Blog · Engineering · 21 October 2026
Open-source models in production: when they win
Frontier APIs and open models are not rivals. A practical way to decide which one runs each step of a workflow.
The debate is often framed as frontier APIs versus open-source models. In production, the best systems use both — and decide step by step.
Where frontier models win
Complex reasoning, long multi-step tasks, nuanced writing and anything where a mistake is expensive. Paying more per call is usually worth it when the step carries the outcome.
Where open models win
- High volume, well-defined steps: classification, extraction, tagging, routing.
- Data that must stay inside: self-hosted models never send a byte outside your environment.
- Predictable cost: a fixed infrastructure bill instead of a per-token one that grows with success.
Decide per step, not per project
A single workflow can use an open model to triage thousands of player reviews, a frontier model to analyze the few that matter, and a small model again to draft the replies. A gateway that routes each request — with evaluation to check that quality holds — turns this from an architecture debate into a configuration choice.
