MIT Press

Table 3:

Accuracy of binary (mono/poly) and multi-class (poly bands) classifiers using SelfSim and pairCos features on the test sets. Comparison to a baseline that predicts always the same class and a classifier that only uses log frequency as feature. Subscripts denote the layers used.

		mono/poly		poly bands
	Model	SelfSim	pairCos	SelfSim	pairCos
en	BERT	0.76₁₀	0.79₈	0.49₁₀	0.46₁₀
	mBERT	0.77₈	0.75₈	0.46₁₂	0.43₁₂
	ELMo	0.69₂	0.63₃	0.37₂	0.34₃
	context2vec	0.61	0.61	0.34	0.31

	Frequency	0.77		0.41

FR	Flaubert	0.58₇	0.55₆	0.29₈	0.27₉
	mBERT	0.66₉	0.64₉	0.38₇	0.38₈

	Frequency	0.61		0.37

ES	BETO	0.70₉	0.66₇	0.42₆	0.48₅
	mBERT	0.69₁₁	0.64₇	0.38₉	0.43₇

	Frequency	0.67		0.41

el	GreekBERT	0.70₄	0.64₄	0.34₄	0.38₆
	mBERT	0.60₇	0.65₇	0.32₁₁	0.34₉

	Frequency	0.63		0.35

	Baseline	0.50		0.25

		mono/poly		poly bands
	Model	SelfSim	pairCos	SelfSim	pairCos
en	BERT	0.76₁₀	0.79₈	0.49₁₀	0.46₁₀
	mBERT	0.77₈	0.75₈	0.46₁₂	0.43₁₂
	ELMo	0.69₂	0.63₃	0.37₂	0.34₃
	context2vec	0.61	0.61	0.34	0.31

	Frequency	0.77		0.41

FR	Flaubert	0.58₇	0.55₆	0.29₈	0.27₉
	mBERT	0.66₉	0.64₉	0.38₇	0.38₈

	Frequency	0.61		0.37

ES	BETO	0.70₉	0.66₇	0.42₆	0.48₅
	mBERT	0.69₁₁	0.64₇	0.38₉	0.43₇

	Frequency	0.67		0.41

el	GreekBERT	0.70₄	0.64₄	0.34₄	0.38₆
	mBERT	0.60₇	0.65₇	0.32₁₁	0.34₉

	Frequency	0.63		0.35

	Baseline	0.50		0.25

View Large

This Feature Is Available To Subscribers Only