LM calibration examples for the T5 model with correct answers in bold. “Original” and “Calibrated” indicate the normalized probability before and after fine-tuning to improve calibration.
| Format . | Input . | Candidate Answers . | Original . | Calibrated . |
|---|---|---|---|---|
| Multiple-choice | Oxygen and sugar are the products of (A) cell division. (B) digestion. (C) photosynthesis. (D) respiration. | cell division. | 0.00 | 0.02 |
| digestion. | 0.00 | 0.01 | ||
| photosynthesis. | 0.00 | 0.83 | ||
| respiration. | 1.00 | 0.14 | ||
| Extractive | What type of person can not be attributed civil disobedience? Civil disobedience is usually defined as pertaining to a citizen’s relation … | head of government | 0.07 | 0.49 |
| public official | 0.91 | 0.26 | ||
| head of government of a country | 0.01 | 0.16 | ||
| public officials | 0.01 | 0.09 |
| Format . | Input . | Candidate Answers . | Original . | Calibrated . |
|---|---|---|---|---|
| Multiple-choice | Oxygen and sugar are the products of (A) cell division. (B) digestion. (C) photosynthesis. (D) respiration. | cell division. | 0.00 | 0.02 |
| digestion. | 0.00 | 0.01 | ||
| photosynthesis. | 0.00 | 0.83 | ||
| respiration. | 1.00 | 0.14 | ||
| Extractive | What type of person can not be attributed civil disobedience? Civil disobedience is usually defined as pertaining to a citizen’s relation … | head of government | 0.07 | 0.49 |
| public official | 0.91 | 0.26 | ||
| head of government of a country | 0.01 | 0.16 | ||
| public officials | 0.01 | 0.09 |