Reasoning / Aggregation
verifiedSelf-Consistency
Ask the same question several times at a temperature above zero, and take the answer that comes up most often. The reasoning paths differ; the correct answer tends to be the one they converge on, because there are many ways to reason correctly to the same place and comparatively few ways to make the same mistake twice.
The distinction from Chain of Thought is what is being aggregated. Chain-of-thought changes what a single sample looks like; this changes how many you take and what you do with them, and it works on top of any prompting scheme that produces a final answer you can compare. It also needs the answer to be extractable and comparable — it works on arithmetic and multiple choice, and does not work on a paragraph, which is the practical limit on where it can be used.
It is a majority vote over samples, so it inherits the majority vote's condition: the correct answer must be the most likely single answer, even if it is well under half. If the model's errors concentrate on one wrong answer more often than they land on the right one, more samples make the result more confidently wrong — voting amplifies the mode, whatever the mode is. Accuracy therefore rises with sample count and saturates at whatever the modal answer's accuracy is, not at one.
extra-samples holds 17% of the budget; rest holds the remaining 83%.
Answers drawn beyond the first, against the single sample a greedy decode would have given, in samples. Drag the sample count up to watch voting take the budget — accuracy saturates at whatever the modal answer's accuracy is, which is not one.
Reviewed by opendroid · 2026-08-18
- arXiv:2203.11171 — Self-Consistency Improves Chain of Thought Reasoning in Language Models
- arXiv:2201.11903 — Chain-of-Thought Prompting Elicits Reasoning in Large Language Models