Evaluation
13 concepts, 13 reviewed and merged.
Capabilities
Methods
- AblationRemove one piece and measure what happens.verified
- Elo RatingRank things by who beats whom rather than by scoring each alone.verified
- Human EvaluationAsk people whether the output is good.verified
- LLM as JudgeGrading open-ended answers needs a human, and humans are slow and expensive.verified
- Statistical SignificanceWhether a difference between two numbers is bigger than the noise that produced them.verified
Metrics
- CalibrationA well-calibrated model is right about 80% of the time when it says it is 80% confident.verified
- pass@kFor code, whether a single answer is right is the wrong question — a developer can generate several attempts and run…verified
- PerplexityPerplexity asks how surprised a model is by text it has not seen.verified
- Precision and RecallTwo questions about the same detector that pull against each other.verified