Calibration and confidence estimation

A calibrated model is one where confidence matches accuracy: when the model says "I am 90% confident," it is correct 90% of the time. Most LLMs are overconfident — they express certainty on questions they answer incorrectly. Calibration measurement is prerequisite to trust.