Overall evaluation results for implicit argument identification. The second column gives the number of ground–truth implicitly filled argument positions for the predicate instances. P, R, and F1 indicate precision, recall, and F–measure (β = 1), respectively. pexact is the bootstrapped exact p-value of the F1 difference between two systems, where the systems are (B)aseline, (D)iscriminative, and (O)racle.
. | Baseline . | Discriminative . | . | Oracle . | . | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
. | # Imp. args. . | P . | R . | F1 . | P . | R . | F1 . | pexact(B,D) . | P . | R . | F1 . | pexact(D,O) . |
| sale | 181 | 57.0 | 27.7 | 37.3 | 59.2 | 44.8 | 51.0 | 0.0003 | 100.0 | 72.4 | 84.0 | ≪0.0001 |
| price | 138 | 67.1 | 23.3 | 34.6 | 56.0 | 48.7 | 52.1 | ≪0.0001 | 100.0 | 78.3 | 87.8 | ≪0.0001 |
| bid | 124 | 66.7 | 14.5 | 23.8 | 60.0 | 36.3 | 45.2 | ≪0.0001 | 100.0 | 60.5 | 75.4 | ≪0.0001 |
| investor | 108 | 30.0 | 2.8 | 5.1 | 46.7 | 39.8 | 43.0 | ≪0.0001 | 100.0 | 84.3 | 91.5 | ≪0.0001 |
| cost | 86 | 60.0 | 10.5 | 17.8 | 62.5 | 50.9 | 56.1 | ≪0.0001 | 100.0 | 86.0 | 92.5 | ≪0.0001 |
| loan | 82 | 63.0 | 20.7 | 31.2 | 67.2 | 50.0 | 57.3 | ≪0.0001 | 100.0 | 89.0 | 94.2 | ≪0.0001 |
| plan | 77 | 72.7 | 20.8 | 32.3 | 59.6 | 44.1 | 50.7 | 0.0032 | 100.0 | 87.0 | 93.1 | ≪0.0001 |
| loss | 62 | 78.8 | 41.9 | 54.7 | 72.5 | 59.7 | 65.5 | 0.0331 | 100.0 | 88.7 | 94.0 | ≪0.0001 |
| fund | 56 | 66.7 | 10.7 | 18.5 | 80.0 | 35.7 | 49.4 | ≪0.0001 | 100.0 | 66.1 | 79.6 | ≪0.0001 |
| investment | 52 | 28.9 | 10.6 | 15.5 | 32.9 | 34.2 | 33.6 | 0.0043 | 100.0 | 80.8 | 89.4 | ≪0.0001 |
| Overall | 966 | 61.4 | 18.9 | 28.9 | 57.9 | 44.5 | 50.3 | ≪0.0001 | 100.0 | 78.0 | 87.6 | ≪0.0001 |
. | Baseline . | Discriminative . | . | Oracle . | . | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
. | # Imp. args. . | P . | R . | F1 . | P . | R . | F1 . | pexact(B,D) . | P . | R . | F1 . | pexact(D,O) . |
| sale | 181 | 57.0 | 27.7 | 37.3 | 59.2 | 44.8 | 51.0 | 0.0003 | 100.0 | 72.4 | 84.0 | ≪0.0001 |
| price | 138 | 67.1 | 23.3 | 34.6 | 56.0 | 48.7 | 52.1 | ≪0.0001 | 100.0 | 78.3 | 87.8 | ≪0.0001 |
| bid | 124 | 66.7 | 14.5 | 23.8 | 60.0 | 36.3 | 45.2 | ≪0.0001 | 100.0 | 60.5 | 75.4 | ≪0.0001 |
| investor | 108 | 30.0 | 2.8 | 5.1 | 46.7 | 39.8 | 43.0 | ≪0.0001 | 100.0 | 84.3 | 91.5 | ≪0.0001 |
| cost | 86 | 60.0 | 10.5 | 17.8 | 62.5 | 50.9 | 56.1 | ≪0.0001 | 100.0 | 86.0 | 92.5 | ≪0.0001 |
| loan | 82 | 63.0 | 20.7 | 31.2 | 67.2 | 50.0 | 57.3 | ≪0.0001 | 100.0 | 89.0 | 94.2 | ≪0.0001 |
| plan | 77 | 72.7 | 20.8 | 32.3 | 59.6 | 44.1 | 50.7 | 0.0032 | 100.0 | 87.0 | 93.1 | ≪0.0001 |
| loss | 62 | 78.8 | 41.9 | 54.7 | 72.5 | 59.7 | 65.5 | 0.0331 | 100.0 | 88.7 | 94.0 | ≪0.0001 |
| fund | 56 | 66.7 | 10.7 | 18.5 | 80.0 | 35.7 | 49.4 | ≪0.0001 | 100.0 | 66.1 | 79.6 | ≪0.0001 |
| investment | 52 | 28.9 | 10.6 | 15.5 | 32.9 | 34.2 | 33.6 | 0.0043 | 100.0 | 80.8 | 89.4 | ≪0.0001 |
| Overall | 966 | 61.4 | 18.9 | 28.9 | 57.9 | 44.5 | 50.3 | ≪0.0001 | 100.0 | 78.0 | 87.6 | ≪0.0001 |