Run and adapt models in Go.

Pre-1.0. Source private today, available for review on request. Working toward a first public v0.x tag.

• Generate, chat, serve — mlx-lm-generate, mlx-lm-chat, and an HTTP server in mlx-lm-serve, from Go commands, with native MLX libraries loaded at runtime.
• Model families — Llama, Gemma, Qwen, DeepSeek, Phi, Mistral, and the rest of the registry, generated via modelir codegen rather than hand-ported one at a time.
• Training and tuning — full-parameter training, LoRA and DoRA adapters, and distillation paths, with text, chat, completion, and preference dataset formats.
• Speculative decoding — draft-model acceleration, with the ANE experiments in mlx-go-ane exploring where the draft stage can run.
• Toolbox commands — quantize, convert, embed, evaluate, perplexity, benchmark, and parity-verification utilities around the core four.

mlx-go supplies arrays and autograd; mlx-go-lm adds model architectures, token generation, serving, evaluation, and training. Those training paths are intended to turn captured usage into an operator-owned model. mlx-go-vibethinker and mlx-go-fedistill both build on them.

Load a checkpoint from a local directory or Hugging Face model ID. Generate text, use it through an HTTP API, or fine-tune an adapter from your own dataset. The server exposes OpenAI- and Anthropic-compatible endpoints for supported requests.

mlx-lm-generate -model ./model -prompt "Summarize this change"
mlx-lm-serve -model ./model
mlx-lm-train -model ./model -train -data ./data -adapter-path ./adapters

Examples from a source build. Apple silicon is the primary host; selected Linux CPU and CUDA workflows are also supported. Model coverage and native runtime requirements depend on the chosen build.

Recent branches improve adapter loading and application before generation, add embedding-model work, and reject non-finite outputs in generation conformance checks. Work also tightens temporary-array cleanup on failed or interrupted generation.

For an integration review, bring the checkpoint, target machine, representative prompts, and any adapter. Those inputs define a useful check of loading, output quality, memory use, and latency.