Train together, one byte per vote.
An implementation of TallyTrain (arXiv:2607.00173): federated distillation where each peer broadcasts a single argmax class index per probe on a shared public set, and everyone learns from the consensus histogram.
Research. Pure array functions for mlx-go, evaluated as an in-process multi-peer simulation. What crosses between peers is votes on public probes (not weights, not raw data) at roughly 115 KB per peer per round in the evaluated setting.
What it does- Vote histograms — builds the consensus distribution from one-byte argmax votes per probe across peers.
- Hybrid objective — the paper's loss: local cross-entropy on labeled probes plus KL divergence against the consensus histogram.
- Consensus decay — linearly curtails the consensus channel over a configurable window, so out-of-distribution probes cannot collapse training.
- Randomized response — an optional differential-privacy mechanism applied to votes before release, giving a formal bound instead of a hope.
- FedAvg bridge — the paper's bandwidth-bridge variant: an optional parameter mean every M rounds for identical architectures.
mlx-go-tallytrain is the lowest-bandwidth rung of the family's federated-training research: where mlx-go-fedistill exchanges weight deltas and mlx-go-ccl synchronizes full training state, tallytrain asks how much peers can learn from each other when all that crosses is a vote. Because the probe set is public and the vote is one byte, the privacy story is legible enough to explain in a sentence, and the randomized-response option makes it a bound rather than a promise.
source private repo, available for review on request — travis@tmc.dev
reference arXiv:2607.00173
contact travis@tmc.dev