Performance of the 11B LMs.
Method . | MC-test . | MT-test . | ||
---|---|---|---|---|
. | ACC . | ECE . | ACC . | ECE . |
T5 | 0.359 | 0.206 | 0.274 | 0.235 |
UnifiedQA | 0.816 | 0.067 | 0.479 | 0.175 |
+ softmax | 0.823 | 0.041 | 0.488 | 0.129 |
+ margin | 0.819 | 0.034 | 0.485 | 0.107 |
+ Temp. | 0.819 | 0.036 | 0.485 | 0.098 |
+ XGB | 0.818 | 0.065 | 0.486 | 0.108 |
+ Para. | 0.820 | 0.035 | 0.484 | 0.092 |
+ Aug. | 0.812 | 0.031 | 0.493 | 0.090 |
+ Combo | 0.807 | 0.032 | 0.494 | 0.085 |
Method . | MC-test . | MT-test . | ||
---|---|---|---|---|
. | ACC . | ECE . | ACC . | ECE . |
T5 | 0.359 | 0.206 | 0.274 | 0.235 |
UnifiedQA | 0.816 | 0.067 | 0.479 | 0.175 |
+ softmax | 0.823 | 0.041 | 0.488 | 0.129 |
+ margin | 0.819 | 0.034 | 0.485 | 0.107 |
+ Temp. | 0.819 | 0.036 | 0.485 | 0.098 |
+ XGB | 0.818 | 0.065 | 0.486 | 0.108 |
+ Para. | 0.820 | 0.035 | 0.484 | 0.092 |
+ Aug. | 0.812 | 0.031 | 0.493 | 0.090 |
+ Combo | 0.807 | 0.032 | 0.494 | 0.085 |