Tool Selection

Given fifty tools and a request, which one — and is the answer none of them? Selection is a retrieval problem before it is a reasoning one, and it degrades in a specific way: past a few dozen tools the model stops reading them all carefully and starts pattern-matching on names.

The scaling fix is the same as everywhere else in this graph — retrieve a shortlist, then decide among it — which turns tool selection into a Dense Retrieval problem over descriptions. The harder failure is the negative case: a model that always picks something will pick wrongly rather than say no tool applies, and that behaviour has to be trained or prompted for rather than assumed.

With n tools in context the selection is an n-way choice whose difficulty grows with description similarity rather than with n itself: two tools that differ in one clause are harder to separate than twenty that do different things. Retrieval reduces n to k before the choice, so accuracy depends on recall@k — a tool that is not shortlisted cannot be selected however well the model reasons.