Table 11 shows the results of Attribute Bagging, compared to the best simple classifier and human agreement (observed agreement, in percentage). The results obtained with AdaBoost (a standard EC; default parameters) are also included as a sanity check. The best results with Attribute Bagging, reported in the table, were obtained using both feature selection and binarization (binarization did not improve results for the remaining classifiers in Tables 10 and 11).
Second model: Results of the ensemble classifiers, compared to the best simple classifier (first row) and to the human agreement on the gold standard (last row). Att. Bagg. stands for Attribute Bagging, and i corresponds to the number of iterations. Percentage human agreement is included in the last row. An FS subscript indicates feature selection, and bin binarization. Columns as in Table 10. Best and second best results are boldfaced. Significant improvements over the best simple classifier are marked as follows: *p < 0.05, **p < 0.01, ***p < 0.001.
. | A: Per-class accuracy . | B: Overall accuracy . | |||
|---|---|---|---|---|---|
. | Qualitative . | Event . | Relational . | Full . | Partial . |
| best simple (all) | 75.5 ± 9.0 | 86.5 ± 6.4 | 86.0 ± 6.5 | 62.5 ± 2.5 | 87.6 ± 2.5 |
| AdaBoost | 82.0* ± 8.6 | 85.6 ± 7.1 | 88.0 ± 6.7 | 66.0* ± 1.9 | 89.9* ± 1.3 |
| Att. Bagg.FS, bin,i=5 | 77.0 ± 8.7 | 85.8 ± 7.1 | 89.0 ± 6.5 | 66.3* ± 1.1 | 87.0 ± 1.5 |
| Att. Bagg.FS, bin,i=100 | 81.0 ± 8.8 | 86.1 ± 6.9 | 90.1* ± 5.3 | 69.1*** ± 1.0 | 89.0 ± 1.0 |
| Human agreement | − | − | − | 68 | 85 |
. | A: Per-class accuracy . | B: Overall accuracy . | |||
|---|---|---|---|---|---|
. | Qualitative . | Event . | Relational . | Full . | Partial . |
| best simple (all) | 75.5 ± 9.0 | 86.5 ± 6.4 | 86.0 ± 6.5 | 62.5 ± 2.5 | 87.6 ± 2.5 |
| AdaBoost | 82.0* ± 8.6 | 85.6 ± 7.1 | 88.0 ± 6.7 | 66.0* ± 1.9 | 89.9* ± 1.3 |
| Att. Bagg.FS, bin,i=5 | 77.0 ± 8.7 | 85.8 ± 7.1 | 89.0 ± 6.5 | 66.3* ± 1.1 | 87.0 ± 1.5 |
| Att. Bagg.FS, bin,i=100 | 81.0 ± 8.8 | 86.1 ± 6.9 | 90.1* ± 5.3 | 69.1*** ± 1.0 | 89.0 ± 1.0 |
| Human agreement | − | − | − | 68 | 85 |