Operator Fusion Pass

Kernel Fusion is the thing being done — merging operations so intermediate values never reach memory. This is the compiler pass that decides which ones to merge, automatically, over a whole graph rather than by hand. The decision is a partition of the graph, and the number of possible partitions is why it is a search problem rather than a rule.

The usual policy is greedy along producer-consumer chains, fusing element-wise operations into whatever heavy operation they follow, and stopping at anything that changes shape or reduces. That heuristic captures most of the available gain and is nowhere near optimal — which is fine, because the remaining gap is smaller than the cost of searching for it in most models.

Partition the graph into regions that will each become one kernel, maximising the intermediate values that stay in registers. The candidate space is exponential in graph size, so every production compiler uses a linear-time greedy policy with hand-written rules about which operator kinds may join — the optimisation is a heuristic and is described as one.