Make every model call earn its keep.

llm-router scores each request on cost and quality, then sends it to a frontier API or a model on hardware you control. It also captures consented traffic as training and evaluation data you keep. One self-hosted binary; no hosted control plane.

Built and tested; publication-pending. Routing and the capture path run today; the distillation classifier and fine-tune training on top are still in build. Source private today, available for review on request.

What it does

Keep frontier calls where their measured quality justifies the price. Move the rest to hardware inside the operator's custody domain. The offline evaluator measures the cost-quality trade; the capture path preserves the consented examples, outcomes, and evaluations needed to train a model later.

Routing and capture run today. Local inference uses mlx-go; distributed-training experiments combine mlx-go-ccl, mlx-go-iroh, and mlx-mesh. The distillation classifier and end-to-end fine-tune loop remain in build. The first owed proof is one scorable task whose unplug result can be rerun with the published evaluation harness.