Provenance / Cost
verifiedEnergy Cost of Training
A training run is a physical event: some number of accelerators drawing some number of watts for some number of weeks. Converting the compute figure everyone quotes into energy is arithmetic, and it is worth doing, because the compute figure is abstract and the energy figure is a quantity people have intuitions about.
The conversion needs three things beyond the FLOP count: the accelerators' power draw, how efficiently the run used them, and the datacentre's overhead for cooling and distribution. That last multiplier is not small and varies by roughly a factor of two across facilities, so an energy figure quoted without it is understated by however much the facility spends on not overheating. Published estimates that do this carefully are rare, and the careful ones disagree with the casual ones by a lot.
Energy is power times time, and both scale with the run, so energy tracks total compute closely — which is the useful simplification. What breaks the proportionality is utilisation: a run at 35% model FLOPs utilisation draws nearly the same power as one at 55% and produces a third less work, so the energy per unit of useful compute is set by the efficiency of the run rather than by its size. That makes utilisation an energy metric as well as a throughput one.
overhead-and-stalls holds 25% of the budget; rest holds the remaining 75%.
Energy a run spends on cooling, distribution and stalled accelerators, against the energy that became useful compute, in equal units. Drag the inefficiency up to watch overhead take the bill — which is why utilisation is an energy figure and not only a throughput one.
Reviewed by opendroid · 2026-08-18
- arXiv:1906.02243 — Energy and Policy Considerations for Deep Learning in NLP
- arXiv:2211.02001 — Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model