Skip to Main Content
Table 2: 

Amnesic probing results for the masked representations. Properties statistics, word-prediction accuracy and DKL results for the different properties inspected in this work. We report the vanilla word prediction accuracy and the Amnesic scores, as well as the Rand and 1-Hot controls which shows minimal information loss and high selectivity (except for the dep property which all information was removed). The DKL is also reported for all properties in the last rows which show similar trends as the accuracy performance.

depf-posc-posnerphrase startphrase end
Properties N. dir 820 675 240 95 35 52 
N. classes 41 45 12 19 2 2 
Majority 11.44 13.22 31.76 86.09 59.25 58.51 
 
Probing Vanilla 71.19 78.32 84.40 90.68 85.53 83.21 
 
LM-Acc Vanilla 56.98 56.98 56.98 57.71 57.71 57.71 
Rand 4.67 24.69 54.55 56.88 57.46 57.27 
Selectivity 20.46 59.51 66.49 60.35 60.97 60.80 
Amnesic 4.67 6.01 33.28 48.39 56.89 56.19 
 
LM-DKL Rand 7.77 6.10 0.45 0.10 0.02 0.04 
Amnesic 7.77 7.26 3.36 1.39 0.06 0.13 
depf-posc-posnerphrase startphrase end
Properties N. dir 820 675 240 95 35 52 
N. classes 41 45 12 19 2 2 
Majority 11.44 13.22 31.76 86.09 59.25 58.51 
 
Probing Vanilla 71.19 78.32 84.40 90.68 85.53 83.21 
 
LM-Acc Vanilla 56.98 56.98 56.98 57.71 57.71 57.71 
Rand 4.67 24.69 54.55 56.88 57.46 57.27 
Selectivity 20.46 59.51 66.49 60.35 60.97 60.80 
Amnesic 4.67 6.01 33.28 48.39 56.89 56.19 
 
LM-DKL Rand 7.77 6.10 0.45 0.10 0.02 0.04 
Amnesic 7.77 7.26 3.36 1.39 0.06 0.13 
View Large
Close Modal
This Feature Is Available To Subscribers Only

or Create an Account

Close Modal
Close Modal
This site uses cookies. By continuing to use our website, you are agreeing to our privacy policy.
Accept