Federated Learning

Train a shared model without collecting the data. Each device trains locally on what it holds, sends an update rather than examples, and a server averages the updates into a new global model. The data never moves, which is the point — and the updates still carry information about it, which is the catch.

The hard parts are statistical and operational rather than cryptographic. Client data is not identically distributed — one phone's typing is not a sample from the population — so local steps drift toward local optima and averaging them is not the same as training centrally. Add unreliable clients, wildly varying hardware, and a communication budget that dominates compute, and the algorithm is mostly about how many local steps to take before averaging.

FedAvg runs E local epochs on each of a sampled fraction of clients, then averages weights in proportion to client data size. Larger E cuts communication rounds and increases client drift, and under non-IID data the averaged model can be worse than any client's — which is why E is the knob that decides whether the method works on a given deployment.