Four groups each contain five toy predictions. Known correct counts are 3, 4, 5, 2, totaling 14/20. Confidence ranges 50..100 in steps of 5, representing P(1); P(0)=100−P(1). Predicted label 1 stays fixed, including the explicit 50/50 tie convention. For this special all-label-1 dataset, positive frequency equals selected-label correctness. General top-label calibration compares selected-label confidence with selected-label correctness.

Bin count is 5 or 10 over the full 0..100 range. Intervals are (lower,upper], except [0,upper] for the first. Every case belongs to exactly one bin, including 100%. Each nonempty bin uses its actual sample mean confidence and correct-count fraction. ECE weights its absolute gap by N/20 and reports percentage points. Empty-bin confidence, accuracy and gap are undefined; zero count gives zero contribution and no plotted point.

Abstention keeps confidence≥threshold, including equality. Coverage divides accepted count by all 20. Retained accuracy divides correct accepted count by accepted count; retained ECE uses the same bin edges with accepted-count weights. With no accepted examples both retained metrics are undefined. The all-20 diagram, full accuracy and all-example ECE remain independent of the threshold.

These finite, hand-authored examples illustrate reporting and aggregation. Manually changing confidence after seeing outcomes is not a fitting or evaluation procedure. Small bins and grouping can hide errors; no confidence intervals, fitted calibrator, representative held-out evidence or population guarantee are supplied. An LLM’s stated confidence is not automatically an empirically calibrated probability; this lesson does not query an LLM or grade generated answers.

Prediction and Reset restore Mixed confidence, 10 bins, threshold 50 and editor D for the current experiment. Actual confidence/bin/threshold edits clear stale answers; inspection and unchanged edits preserve them. Free-exploration Reset begins Experiment 1.