Disagreement on identifier type, factored by entity type. The numbers in parenthesis denote the percentage of mentions marked by two annotators that have this type of disagreement.
| Entity type . | DIRECT vs. QUASI . | DIRECT vs. NO_MASK . | QUASI vs. NO_MASK . |
|---|---|---|---|
| CODE | 65 (1.5) | 0 (0.0) | 70 (1.6) |
| DATETIME | 16 (0.4) | 0 (0.0) | 1,000 (23.3) |
| DEM | 0 (0.0) | 0 (0.0) | 497 (11.6) |
| LOC | 0 (0.0) | 0 (0.0) | 416 (9.7) |
| MISC | 0 (0.0) | 0 (0.0) | 158 (3.7) |
| ORG | 5 (0.1) | 0 (0.0) | 1,068 (24.8) |
| PERSON | 151 (3.5) | 0 (0.0) | 752 (17.5) |
| QUANTITY | 0 (0.0) | 0 (0.0) | 101 (2.4) |
| Total | 237 | 0 (0.0) | 4,062 |
| Entity type . | DIRECT vs. QUASI . | DIRECT vs. NO_MASK . | QUASI vs. NO_MASK . |
|---|---|---|---|
| CODE | 65 (1.5) | 0 (0.0) | 70 (1.6) |
| DATETIME | 16 (0.4) | 0 (0.0) | 1,000 (23.3) |
| DEM | 0 (0.0) | 0 (0.0) | 497 (11.6) |
| LOC | 0 (0.0) | 0 (0.0) | 416 (9.7) |
| MISC | 0 (0.0) | 0 (0.0) | 158 (3.7) |
| ORG | 5 (0.1) | 0 (0.0) | 1,068 (24.8) |
| PERSON | 151 (3.5) | 0 (0.0) | 752 (17.5) |
| QUANTITY | 0 (0.0) | 0 (0.0) | 101 (2.4) |
| Total | 237 | 0 (0.0) | 4,062 |