Train together, one byte per vote.
An implementation of TallyTrain (arXiv:2607.00173): federated distillation where each peer broadcasts a single argmax class index per probe on a shared public set, and everyone learns from the consensus histogram.
Research. Pure array functions for mlx-go, evaluated as an in-process multi-peer simulation. What crosses between peers is votes on public probes (not weights, not raw data) at roughly 115 KB per peer per round in the evaluated setting.
What it does- Vote histograms — builds the consensus distribution from one-byte argmax votes per probe across peers.
- Hybrid objective — the paper's loss: local cross-entropy on labeled probes plus KL divergence against the consensus histogram.
- Consensus decay — linearly curtails the consensus channel over a configurable window, so out-of-distribution probes cannot collapse training.
- Randomized response — an optional differential-privacy mechanism applied to votes before release, giving a formal bound instead of a hope.
- FedAvg bridge — the paper's bandwidth-bridge variant: an optional parameter mean every M rounds for identical architectures.
mlx-go-fedistill exchanges weight deltas, and mlx-go-ccl synchronizes fuller training state. TallyTrain sends one class vote per public probe. In the evaluated setting that is roughly 115 KB per peer per round; optional randomized response adds a stated differential-privacy bound to those votes.
source private repo, available for review on request — travis@tmc.dev
reference arXiv:2607.00173
contact travis@tmc.dev