Route work to the Neural Engine.
Experimental execution paths that move selected mlx-go work (linear operations, draft models, decode stages) onto the Apple Neural Engine, with CPU and GPU fallback preserved.
Experiment. Intentionally narrow and not API-stable, in the repo's own words. Model assessment and routing helpers build everywhere; the block runtimes and IOSurface data plane are Darwin-only and require private-ANE build tags.
What it does- Model assessment — classifies whether a model can reuse the existing ANE draft and decode paths or needs a model-specific backend.
- Linear routing — intercepts
nn.Linearforward passes via an optional hook and evaluates them on the ANE when input shapes meet the measured constraints. - Block runtimes — standalone draft-model and decode-stage targets for offloading whole transformer stages in speculative decoding loops.
- IOSurface data plane — plan-owned IOSurfaces, shared Metal events, and reusable buffer bindings for zero-copy handoff between GPU and ANE.
- Deterministic fallback — anything the router declines runs on the normal mlx-go CPU/GPU path, unchanged.
This experiment intercepts recognized mlx-go subgraphs and sends matching operations to the Neural Engine. It does not yet lower arbitrary graphs. Results from mlx-go-lm draft-model and decode experiments are reported as implementation findings, not rules about every ANE workload; gputrace measures the GPU side of the comparison.