Skip to Main Content
Table 8

Classification accuracy on the probing tasks for the 8 non-contextualized DSMs.

TaskPPMISGNSGloVeFastTextMajority
w2synfsyntw2synfsynt
Positive/Negative 0.52 0.65 0.65 0.83 0.85 0.79 0.79 1.00 0.52 
Concrete/Abstract 0.69 0.70 0.73 1.00 1.00 1.00 1.00 1.00 0.59 
Animate/Inanimate 0.74 0.78 0.86 0.94 0.96 0.97 0.96 0.98 0.83 
VerbNet 
pour-9.5+spray-9.7 0.61 0.52 0.48 0.70 0.78 0.78 0.82 0.74 0.52 
fill-9.8 0.50 0.50 0.52 0.74 0.72 0.74 0.70 0.74 0.50 
butter-9.9 0.47 0.55 0.50 0.87 0.84 0.86 0.89 0.89 0.63 
pocket-9.10 0.58 0.50 0.50 0.88 0.88 0.92 0.75 0.83 0.50 
remove-10.1+clear-10.3 0.44 0.56 0.44 0.56 0.68 0.76 0.68 0.64 0.52 
 +mine-10.9 
steal-10.5 0.48 0.48 0.41 0.83 0.83 0.79 0.86 0.90 0.52 
debone-10.8 0.59 0.63 0.54 0.86 0.82 0.82 0.68 0.90 0.50 
cut-21.1+carve-21.2 0.65 0.65 0.57 0.80 0.80 0.96 0.57 0.73 0.50 
amalgamate-22.2 0.56 0.52 0.52 0.74 0.70 0.78 0.74 0.83 0.52 
tape-22.4 0.65 0.62 0.73 0.98 0.97 0.94 0.98 1.00 0.59 
characterize-29.2 0.57 0.57 0.57 0.81 0.81 0.86 0.76 0.81 0.52 
amuse-31.1 0.55 0.56 0.51 0.69 0.80 0.75 0.67 0.72 0.51 
admire-31.2 0.44 0.52 0.56 0.87 0.91 0.87 0.78 0.96 0.52 
marvel-31.3 0.59 0.62 0.65 0.72 0.69 0.83 0.69 0.76 0.58 
judgement-33.1 0.71 0.66 0.66 0.77 0.80 0.80 0.77 0.80 0.54 
manner_of_speaking-37.3 0.71 0.79 0.48 0.79 0.86 0.90 0.90 0.86 0.50 
say-37.7 0.62 0.60 0.50 0.55 0.50 0.64 0.50 0.55 0.55 
animal_sounds-38 0.67 0.80 0.67 0.87 0.90 0.83 0.83 0.93 0.56 
sound_emission-43.2 0.52 0.48 0.61 0.70 0.78 0.74 0.65 0.70 0.52 
cooking-45.3 0.55 0.60 0.45 0.77 0.86 0.90 0.73 0.86 0.52 
other_cos-45.4 0.54 0.50 0.57 0.70 0.73 0.74 0.76 0.68 0.50 
contiguous_location-47.8 0.57 0.57 0.52 0.86 0.76 0.81 0.61 0.81 0.52 
run-51.3.2 0.52 0.48 0.56 0.80 0.84 0.80 0.75 0.80 0.56 
TaskPPMISGNSGloVeFastTextMajority
w2synfsyntw2synfsynt
Positive/Negative 0.52 0.65 0.65 0.83 0.85 0.79 0.79 1.00 0.52 
Concrete/Abstract 0.69 0.70 0.73 1.00 1.00 1.00 1.00 1.00 0.59 
Animate/Inanimate 0.74 0.78 0.86 0.94 0.96 0.97 0.96 0.98 0.83 
VerbNet 
pour-9.5+spray-9.7 0.61 0.52 0.48 0.70 0.78 0.78 0.82 0.74 0.52 
fill-9.8 0.50 0.50 0.52 0.74 0.72 0.74 0.70 0.74 0.50 
butter-9.9 0.47 0.55 0.50 0.87 0.84 0.86 0.89 0.89 0.63 
pocket-9.10 0.58 0.50 0.50 0.88 0.88 0.92 0.75 0.83 0.50 
remove-10.1+clear-10.3 0.44 0.56 0.44 0.56 0.68 0.76 0.68 0.64 0.52 
 +mine-10.9 
steal-10.5 0.48 0.48 0.41 0.83 0.83 0.79 0.86 0.90 0.52 
debone-10.8 0.59 0.63 0.54 0.86 0.82 0.82 0.68 0.90 0.50 
cut-21.1+carve-21.2 0.65 0.65 0.57 0.80 0.80 0.96 0.57 0.73 0.50 
amalgamate-22.2 0.56 0.52 0.52 0.74 0.70 0.78 0.74 0.83 0.52 
tape-22.4 0.65 0.62 0.73 0.98 0.97 0.94 0.98 1.00 0.59 
characterize-29.2 0.57 0.57 0.57 0.81 0.81 0.86 0.76 0.81 0.52 
amuse-31.1 0.55 0.56 0.51 0.69 0.80 0.75 0.67 0.72 0.51 
admire-31.2 0.44 0.52 0.56 0.87 0.91 0.87 0.78 0.96 0.52 
marvel-31.3 0.59 0.62 0.65 0.72 0.69 0.83 0.69 0.76 0.58 
judgement-33.1 0.71 0.66 0.66 0.77 0.80 0.80 0.77 0.80 0.54 
manner_of_speaking-37.3 0.71 0.79 0.48 0.79 0.86 0.90 0.90 0.86 0.50 
say-37.7 0.62 0.60 0.50 0.55 0.50 0.64 0.50 0.55 0.55 
animal_sounds-38 0.67 0.80 0.67 0.87 0.90 0.83 0.83 0.93 0.56 
sound_emission-43.2 0.52 0.48 0.61 0.70 0.78 0.74 0.65 0.70 0.52 
cooking-45.3 0.55 0.60 0.45 0.77 0.86 0.90 0.73 0.86 0.52 
other_cos-45.4 0.54 0.50 0.57 0.70 0.73 0.74 0.76 0.68 0.50 
contiguous_location-47.8 0.57 0.57 0.52 0.86 0.76 0.81 0.61 0.81 0.52 
run-51.3.2 0.52 0.48 0.56 0.80 0.84 0.80 0.75 0.80 0.56 
Close Modal

or Create an Account

Close Modal
Close Modal