Prediction F1 breakdown for all models on the DocBank dataset.
. | Abstract . | Author . | Caption . | Date . | Figure . | Footer . | List . | Paragraph . | Reference . | Section . | Table . | Title . | Macro F1 . |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
BERTBASE | 97.82 | 89.96 | 93.91 | 87.33 | 71.97 | 84.76 | 75.99 | 96.84 | 92.05 | 92.81 | 74.19 | 89.31 | 87.24 |
BERTBASE + I-VILA(Text Line) | 97.99 | 90.67 | 95.74 | 88.12 | 88.85 | 88.29 | 80.20 | 97.85 | 92.68 | 94.91 | 77.39 | 90.34 | 90.25 |
BERTBASE + I-VILA(Text Block) | 98.15 | 90.66 | 96.56 | 87.83 | 79.49 | 88.40 | 80.72 | 97.51 | 92.62 | 94.86 | 76.91 | 90.22 | 89.49 |
LayoutLMBASE | 98.63 | 92.25 | 96.88 | 87.13 | 76.56 | 94.26 | 89.67 | 97.72 | 93.16 | 96.31 | 77.38 | 92.80 | 91.06 |
LayoutLMBASE + Sentence Breaks | 98.48 | 92.70 | 96.93 | 88.06 | 77.65 | 94.35 | 90.46 | 97.81 | 92.61 | 96.58 | 78.84 | 92.81 | 91.44 |
LayoutLMBASE + I-VILA(Text Line) | 98.57 | 92.64 | 97.35 | 87.87 | 90.78 | 94.37 | 90.77 | 98.44 | 92.87 | 96.60 | 80.43 | 92.78 | 92.79 |
LayoutLMBASE + I-VILA(Text Block) | 98.68 | 92.31 | 97.44 | 87.69 | 83.41 | 94.03 | 90.56 | 98.13 | 93.27 | 96.44 | 79.51 | 92.48 | 92.00 |
LayoutLMv2BASE | 98.68 | 93.04 | 97.49 | 89.55 | 85.60 | 95.30 | 93.63 | 98.46 | 94.30 | 96.48 | 84.41 | 93.10 | 93.34 |
Simple Group Classifier | 93.85 | 84.68 | 96.55 | 71.04 | 80.63 | 91.58 | 83.84 | 97.53 | 92.54 | 85.33 | 73.85 | 92.65 | 87.01 |
H-VILA(Text Line) | 98.68 | 90.95 | 95.46 | 80.99 | 88.79 | 93.84 | 90.77 | 98.36 | 93.81 | 95.27 | 78.46 | 89.81 | 91.27 |
H-VILA(Text Block) | 98.57 | 86.81 | 95.76 | 70.33 | 80.29 | 91.23 | 79.82 | 97.53 | 92.97 | 86.70 | 79.84 | 93.52 | 87.78 |
# Tokens in Class | 461898 | 81061 | 858862 | 3275 | 932150 | 158176 | 684786 | 20630188 | 1813594 | 154062 | 235801 | 26355 | – |
. | Abstract . | Author . | Caption . | Date . | Figure . | Footer . | List . | Paragraph . | Reference . | Section . | Table . | Title . | Macro F1 . |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
BERTBASE | 97.82 | 89.96 | 93.91 | 87.33 | 71.97 | 84.76 | 75.99 | 96.84 | 92.05 | 92.81 | 74.19 | 89.31 | 87.24 |
BERTBASE + I-VILA(Text Line) | 97.99 | 90.67 | 95.74 | 88.12 | 88.85 | 88.29 | 80.20 | 97.85 | 92.68 | 94.91 | 77.39 | 90.34 | 90.25 |
BERTBASE + I-VILA(Text Block) | 98.15 | 90.66 | 96.56 | 87.83 | 79.49 | 88.40 | 80.72 | 97.51 | 92.62 | 94.86 | 76.91 | 90.22 | 89.49 |
LayoutLMBASE | 98.63 | 92.25 | 96.88 | 87.13 | 76.56 | 94.26 | 89.67 | 97.72 | 93.16 | 96.31 | 77.38 | 92.80 | 91.06 |
LayoutLMBASE + Sentence Breaks | 98.48 | 92.70 | 96.93 | 88.06 | 77.65 | 94.35 | 90.46 | 97.81 | 92.61 | 96.58 | 78.84 | 92.81 | 91.44 |
LayoutLMBASE + I-VILA(Text Line) | 98.57 | 92.64 | 97.35 | 87.87 | 90.78 | 94.37 | 90.77 | 98.44 | 92.87 | 96.60 | 80.43 | 92.78 | 92.79 |
LayoutLMBASE + I-VILA(Text Block) | 98.68 | 92.31 | 97.44 | 87.69 | 83.41 | 94.03 | 90.56 | 98.13 | 93.27 | 96.44 | 79.51 | 92.48 | 92.00 |
LayoutLMv2BASE | 98.68 | 93.04 | 97.49 | 89.55 | 85.60 | 95.30 | 93.63 | 98.46 | 94.30 | 96.48 | 84.41 | 93.10 | 93.34 |
Simple Group Classifier | 93.85 | 84.68 | 96.55 | 71.04 | 80.63 | 91.58 | 83.84 | 97.53 | 92.54 | 85.33 | 73.85 | 92.65 | 87.01 |
H-VILA(Text Line) | 98.68 | 90.95 | 95.46 | 80.99 | 88.79 | 93.84 | 90.77 | 98.36 | 93.81 | 95.27 | 78.46 | 89.81 | 91.27 |
H-VILA(Text Block) | 98.57 | 86.81 | 95.76 | 70.33 | 80.29 | 91.23 | 79.82 | 97.53 | 92.97 | 86.70 | 79.84 | 93.52 | 87.78 |
# Tokens in Class | 461898 | 81061 | 858862 | 3275 | 932150 | 158176 | 684786 | 20630188 | 1813594 | 154062 | 235801 | 26355 | – |