Memory Planning

Intermediate tensors have lifetimes, and two that never coexist can share the same memory. Planning that reuse is what lets a model with far more intermediate values than memory run at all, and it is decided at compile time from the graph rather than by an allocator at runtime.

Static planning beats dynamic allocation because it can see the whole schedule, and it interacts with everything else: fusion removes intermediates entirely, checkpointing trades memory for recomputation, and changing the execution order changes which lifetimes overlap. Peak memory is a property of the schedule rather than of the model, which is why the same model fits on a device under one compiler and not another.

Given each tensor's live interval, the problem is interval-graph colouring — assign offsets so that overlapping intervals never share memory, minimising the peak. The lower bound is the maximum total size of simultaneously live tensors, and a good planner gets close to it; the gap is fragmentation rather than ignorance.