Prediction F1 breakdown for all models on the S2-VL dataset. Similar to the results in the main paper, we show averaged scores with standard deviation in parentheses across the 5-fold cross validation subsets.
| . | Abstract . | Author . | Bibliography . | Caption . | Equation . | Figure . | Footer . | Footnote . |
|---|---|---|---|---|---|---|---|---|
| BERTBASE | 91.67(5.51) | 71.38(18.79) | 97.90(1.59) | 94.64(1.38) | 76.23(4.36) | 60.14(24.13) | 61.99(17.04) | 62.91(7.23) |
| BERTBASE + I-VILA(Text Line) | 89.38(6.50) | 65.93(15.48) | 97.92(1.56) | 96.66(1.39) | 83.22(5.87) | 72.11(13.35) | 57.75(22.46) | 72.78(12.45) |
| BERTBASE + I-VILA(Text Block) | 90.45(3.61) | 64.97(16.11) | 97.21(1.27) | 96.82(0.94) | 83.56(5.59) | 70.57(11.56) | 59.79(23.18) | 80.17(10.48) |
| LayoutLMBASE | 91.87(4.89) | 69.39(11.30) | 98.08(1.13) | 92.98(7.35) | 77.49(7.13) | 74.46(18.48) | 67.42(18.90) | 77.22(17.59) |
| LayoutLMBASE + Sentence Breaks | 92.01(4.79) | 69.22(11.02) | 98.57(1.24) | 95.74(1.36) | 77.94(9.68) | 67.80(25.61) | 69.67(20.06) | 78.57(16.45) |
| LayoutLMBASE + I-VILA(Text Line) | 91.77(5.85) | 69.81(7.86) | 98.09(1.64) | 94.06(2.91) | 84.48(7.00) | 71.57(21.49) | 67.23(23.01) | 77.10(15.64) |
| LayoutLMBASE + I-VILA(Text Block) | 92.91(4.02) | 70.42(13.38) | 98.19(1.57) | 97.19(1.16) | 83.76(6.61) | 68.38(26.11) | 68.03(19.11) | 76.77(17.64) |
| LayoutLMv2BASE | 91.09(6.46) | 63.42(17.55) | 97.74(2.00) | 96.73(1.39) | 77.18(13.70) | 83.71(11.53) | 64.37(22.24) | 70.20(12.43) |
| H-VILA(Text Line) | 93.90(5.16) | 70.86(9.78) | 97.71(1.26) | 92.86(3.89) | 81.38(7.79) | 77.86(10.65) | 65.95(23.44) | 81.76(15.03) |
| H-VILA(Text Block) | 93.40(6.14) | 67.03(19.43) | 96.11(3.38) | 92.76(6.47) | 86.87(8.64) | 79.64(11.21) | 63.72(22.01) | 83.66(9.88) |
| # Tokens in Class | 2854(432) | 543(118) | 15681(3704) | 4046(2119) | 2552(1872) | 1402(1316) | 480(205) | 2468(1254) |
| contd. | Header | Keywords | List | Paragraph | Section | Table | Title | Macro F1 |
| BERTBASE | 76.47(8.51) | 90.16(6.44) | 51.00(16.90) | 96.07(1.37) | 79.72(3.46) | 79.93(16.26) | 84.81(8.52) | 78.34(6.53) |
| BERTBASE + I-VILA(Text Line) | 81.53(7.94) | 87.06(5.57) | 58.64(8.10) | 96.67(1.13) | 87.21(3.25) | 85.58(15.67) | 84.80(5.84) | 81.15(4.83) |
| BERTBASE + I-VILA(Text Block) | 83.99(8.74) | 87.86(7.51) | 62.01(13.25) | 96.65(1.21) | 86.71(3.23) | 80.44(16.35) | 86.14(5.23) | 81.82(4.88) |
| LayoutLMBASE | 88.21(5.81) | 88.14(5.94) | 58.21(15.15) | 96.88(0.87) | 88.14(2.73) | 82.02(15.58) | 89.90(8.17) | 82.69(6.04) |
| LayoutLMBASE + Sentence Breaks | 88.08(5.71) | 88.80(3.23) | 60.61(11.80) | 97.01(0.85) | 88.05(2.79) | 81.59(16.22) | 88.52(5.92) | 82.81(5.21) |
| LayoutLMBASE + I-VILA(Text Line) | 87.14(6.49) | 86.66(6.24) | 65.82(10.92) | 97.17(1.26) | 89.79(2.48) | 86.00(12.33) | 89.89(7.47) | 83.77(5.75) |
| LayoutLMBASE + I-VILA(Text Block) | 88.39(6.20) | 90.92(3.97) | 59.06(17.99) | 97.17(1.14) | 88.67(3.57) | 81.84(15.77) | 89.95(6.32) | 83.44(6.48) |
| LayoutLMv2BASE | 86.95(6.84) | 89.71(7.95) | 68.36(10.05) | 96.65(0.71) | 89.48(4.13) | 81.69(15.05) | 88.46(6.00) | 83.05(4.51) |
| H-VILA(Text Line) | 87.89(6.45) | 86.34(5.02) | 65.76(10.26) | 96.90(0.75) | 85.45(2.02) | 85.19(7.55) | 85.62(6.00) | 83.69(2.92) |
| H-VILA(Text Block) | 86.49(6.08) | 76.97(18.82) | 55.82(16.99) | 96.43(1.40) | 86.72(4.55) | 81.38(14.94) | 84.39(9.10) | 82.09(5.89) |
| # Tokens in Class | 1122(463) | 130(27) | 2274(593) | 95732(8226) | 882(113) | 3887(2041) | 240(26) | – |
| . | Abstract . | Author . | Bibliography . | Caption . | Equation . | Figure . | Footer . | Footnote . |
|---|---|---|---|---|---|---|---|---|
| BERTBASE | 91.67(5.51) | 71.38(18.79) | 97.90(1.59) | 94.64(1.38) | 76.23(4.36) | 60.14(24.13) | 61.99(17.04) | 62.91(7.23) |
| BERTBASE + I-VILA(Text Line) | 89.38(6.50) | 65.93(15.48) | 97.92(1.56) | 96.66(1.39) | 83.22(5.87) | 72.11(13.35) | 57.75(22.46) | 72.78(12.45) |
| BERTBASE + I-VILA(Text Block) | 90.45(3.61) | 64.97(16.11) | 97.21(1.27) | 96.82(0.94) | 83.56(5.59) | 70.57(11.56) | 59.79(23.18) | 80.17(10.48) |
| LayoutLMBASE | 91.87(4.89) | 69.39(11.30) | 98.08(1.13) | 92.98(7.35) | 77.49(7.13) | 74.46(18.48) | 67.42(18.90) | 77.22(17.59) |
| LayoutLMBASE + Sentence Breaks | 92.01(4.79) | 69.22(11.02) | 98.57(1.24) | 95.74(1.36) | 77.94(9.68) | 67.80(25.61) | 69.67(20.06) | 78.57(16.45) |
| LayoutLMBASE + I-VILA(Text Line) | 91.77(5.85) | 69.81(7.86) | 98.09(1.64) | 94.06(2.91) | 84.48(7.00) | 71.57(21.49) | 67.23(23.01) | 77.10(15.64) |
| LayoutLMBASE + I-VILA(Text Block) | 92.91(4.02) | 70.42(13.38) | 98.19(1.57) | 97.19(1.16) | 83.76(6.61) | 68.38(26.11) | 68.03(19.11) | 76.77(17.64) |
| LayoutLMv2BASE | 91.09(6.46) | 63.42(17.55) | 97.74(2.00) | 96.73(1.39) | 77.18(13.70) | 83.71(11.53) | 64.37(22.24) | 70.20(12.43) |
| H-VILA(Text Line) | 93.90(5.16) | 70.86(9.78) | 97.71(1.26) | 92.86(3.89) | 81.38(7.79) | 77.86(10.65) | 65.95(23.44) | 81.76(15.03) |
| H-VILA(Text Block) | 93.40(6.14) | 67.03(19.43) | 96.11(3.38) | 92.76(6.47) | 86.87(8.64) | 79.64(11.21) | 63.72(22.01) | 83.66(9.88) |
| # Tokens in Class | 2854(432) | 543(118) | 15681(3704) | 4046(2119) | 2552(1872) | 1402(1316) | 480(205) | 2468(1254) |
| contd. | Header | Keywords | List | Paragraph | Section | Table | Title | Macro F1 |
| BERTBASE | 76.47(8.51) | 90.16(6.44) | 51.00(16.90) | 96.07(1.37) | 79.72(3.46) | 79.93(16.26) | 84.81(8.52) | 78.34(6.53) |
| BERTBASE + I-VILA(Text Line) | 81.53(7.94) | 87.06(5.57) | 58.64(8.10) | 96.67(1.13) | 87.21(3.25) | 85.58(15.67) | 84.80(5.84) | 81.15(4.83) |
| BERTBASE + I-VILA(Text Block) | 83.99(8.74) | 87.86(7.51) | 62.01(13.25) | 96.65(1.21) | 86.71(3.23) | 80.44(16.35) | 86.14(5.23) | 81.82(4.88) |
| LayoutLMBASE | 88.21(5.81) | 88.14(5.94) | 58.21(15.15) | 96.88(0.87) | 88.14(2.73) | 82.02(15.58) | 89.90(8.17) | 82.69(6.04) |
| LayoutLMBASE + Sentence Breaks | 88.08(5.71) | 88.80(3.23) | 60.61(11.80) | 97.01(0.85) | 88.05(2.79) | 81.59(16.22) | 88.52(5.92) | 82.81(5.21) |
| LayoutLMBASE + I-VILA(Text Line) | 87.14(6.49) | 86.66(6.24) | 65.82(10.92) | 97.17(1.26) | 89.79(2.48) | 86.00(12.33) | 89.89(7.47) | 83.77(5.75) |
| LayoutLMBASE + I-VILA(Text Block) | 88.39(6.20) | 90.92(3.97) | 59.06(17.99) | 97.17(1.14) | 88.67(3.57) | 81.84(15.77) | 89.95(6.32) | 83.44(6.48) |
| LayoutLMv2BASE | 86.95(6.84) | 89.71(7.95) | 68.36(10.05) | 96.65(0.71) | 89.48(4.13) | 81.69(15.05) | 88.46(6.00) | 83.05(4.51) |
| H-VILA(Text Line) | 87.89(6.45) | 86.34(5.02) | 65.76(10.26) | 96.90(0.75) | 85.45(2.02) | 85.19(7.55) | 85.62(6.00) | 83.69(2.92) |
| H-VILA(Text Block) | 86.49(6.08) | 76.97(18.82) | 55.82(16.99) | 96.43(1.40) | 86.72(4.55) | 81.38(14.94) | 84.39(9.10) | 82.09(5.89) |
| # Tokens in Class | 1122(463) | 130(27) | 2274(593) | 95732(8226) | 882(113) | 3887(2041) | 240(26) | – |