Function Calling

A model that can only produce text cannot check the weather. Function calling hands it a list of things it may do and lets it answer with a structured request instead of prose, which the surrounding program executes and hands back. The model never runs anything; it asks, and something else decides whether to comply.

The tool list goes in the context, the model emits a name and arguments, and the runtime validates and dispatches. Everything the model knows about a tool is what its description says, so a bad description is a bug in the model's behaviour with no stack trace and no line number. The output has to be machine-parseable, which is why this pairs with constrained decoding rather than with hoping.

With t tools whose schemas cost d tokens each, t·d is paid on every request whether or not a tool is used. The model chooses among them by description alone, so two tools whose descriptions overlap are exactly as hard to tell apart as their text is — the limit is distinguishability rather than context length.