Can two operator-controlled machines contribute to one model without a datacenter coordinator? We tested the smallest useful case: two consumer Apple Silicon machines, Qwen3-0.6B-4bit, a network path between the hosts, and three adapter-exchange rounds.
The run
Each machine independently fine-tuned a rank-2 LoRA adapter on Qwen3-0.6B-4bit. After each round, the machines exchanged adapters over transport built on mlx-go-iroh and combined them with the same merge rule. Three rounds completed. Both hosts wrote a byte-identical merged adapter after every round, and validation loss fell from 6.07 to 2.23 without collapsing to zero.
Why byte-identical is the interesting part
Byte equality of a floating-point computation on two independent machines is not the default outcome. Floating-point addition is not associative; summation order, fused multiply-add, parallel reduction, and serialization metadata all change bytes. Getting the same bytes on both hosts means the merge behaves like a miniature reproducible build: operands in, one canonical file out, on any conforming machine. That turns the merged-adapter hash into an agreement object. A round succeeds when both hosts write the same hash (the recorded rounds: b1f9613c, 3c499c33, 1ca76dfb), and any disagreement is a signal, not a shrug. The conditions under which this property holds, and the ways real implementations silently break it, are worked out in a companion analysis with a pre-registered cross-architecture test plan.
The failure ladder
Transport dominated the run. Of one 95-second round, 93.8 seconds went to pushing 2.5 MB through a whole-value store over a QUIC relay, an effective rate near 0.027 MB/s. Base64 expansion stalled QUIC flow control between the two hosts although the same test passed on loopback. A rank-4 attempt with a 10 MB adapter then failed on the relay path before merge. That is a transport-scale failure, not a convergence result.
The fix is designed, not hand-waved: a streaming binary-frame store protocol with resumable verified transfers, an analytical latency model calibrated against this run, and pre-registered pass bars. The 10 MB case that failed must complete, and the effective rate must beat the measured workaround by at least five times, or the design is wrong.
What this does not show
The run used two controlled machines, a 0.6B-parameter model, rank-2 adapters, three rounds, and no adversarial peers. It does not demonstrate a swarm, permissionless participation, or coordinated group training. The end-to-end replay harness is also still owed. It does demonstrate adapter exchange over a physical network and byte agreement on the merged output.
The full write-ups, with run records, are in preparation for publication. The transport is public today: mlx-go-iroh over go-iroh.