Extractability measures in the different models we inspect. Best model for each measure highlighted in bold.
| Model . | Succ-Patt . | Succ-Objs . | Unk-Const . | Know-Const . |
|---|---|---|---|---|
| majority | 97.3±7.3 | 23.2±21.0 | 100.0±0.0 | 100.0±0.0 |
| BERT-base | 100.0 ±0.0 | 63.0±19.9 | 46.5±21.7 | 63.8±24.5 |
| BERT-large | 100.0 ±0.0 | 65.7 ±22.1 | 48.1±20.2 | 65.2±23.8 |
| BERT-large-wwm | 100.0 ±0.0 | 64.9±20.3 | 49.5 ±20.1 | 65.3 ±25.1 |
| RoBERTa-base | 100.0 ±0.0 | 56.2±22.7 | 43.9±15.8 | 56.3±19.0 |
| RoBERTa-large | 100.0 ±0.0 | 60.1±22.3 | 46.8±18.0 | 60.5±21.1 |
| ALBERT-base | 100.0 ±0.0 | 45.8±23.7 | 41.4±17.3 | 56.3±22.0 |
| ALBERT-xxlarge | 100.0 ±0.0 | 58.8±23.8 | 40.5±16.4 | 57.5±23.8 |
| Model . | Succ-Patt . | Succ-Objs . | Unk-Const . | Know-Const . |
|---|---|---|---|---|
| majority | 97.3±7.3 | 23.2±21.0 | 100.0±0.0 | 100.0±0.0 |
| BERT-base | 100.0 ±0.0 | 63.0±19.9 | 46.5±21.7 | 63.8±24.5 |
| BERT-large | 100.0 ±0.0 | 65.7 ±22.1 | 48.1±20.2 | 65.2±23.8 |
| BERT-large-wwm | 100.0 ±0.0 | 64.9±20.3 | 49.5 ±20.1 | 65.3 ±25.1 |
| RoBERTa-base | 100.0 ±0.0 | 56.2±22.7 | 43.9±15.8 | 56.3±19.0 |
| RoBERTa-large | 100.0 ±0.0 | 60.1±22.3 | 46.8±18.0 | 60.5±21.1 |
| ALBERT-base | 100.0 ±0.0 | 45.8±23.7 | 41.4±17.3 | 56.3±22.0 |
| ALBERT-xxlarge | 100.0 ±0.0 | 58.8±23.8 | 40.5±16.4 | 57.5±23.8 |