A schematic description of the proposed amnesic intervention: We transform the contextualized representation of the word “ran” so as to remove information (here, POS), resulting in a “cleaned” version $hran¬POS$⁠. This representation is fed to the word-prediction layer and the behavioral influence of POS erasure is measured.