Table 1:
Final memory states for a fully trained MMU network across all unique permutations of the 3-deep Sequence Classification task (number of distractors set to a constant value of 15).
#PermutationSumFinal Memory State[M2, M3, M5]Signature
[$-1,-1,-$1] $-$[$-0.33,-$1.51, 2.72, 0.84, $-$0.80] [$-$1.51, 2.72, $-$0.80] [0,2,1]
[$-1,-$1,1] $-$[$-0.33,-$1.51, 1.73, 0.84, $-$1.46] [$-$1.51, 1.73, $-$1.46] [0,2,1]
[$-1,1,-$1] $-$[$-0.33,-$1.51, 1.73, 0.84, $-$1.46] [$-$1.51, 1.73, $-$1.46] [0,2,1]
[1,$-1,-$1] $-$[$-0.33,-$1.51, 1.73, 0.84, $-$1.46] [$-$1.51, 1.73, $-$1.46] [0,2,1]
[$-$1,1,1] $+$[$-0.33,-$1.51, 0.78, 0.84, $-$2.36] [$-$1.51, 0.78, $-$2.36] [2,0,1]
[1,$-$1,1] $+$[$-0.33,-$1.51, 0.78, 0.84, $-$2.36] [$-$1.51, 0.78, $-$2.36] [2,0,1]
[1,1,$-$1] $+$[$-0.33,-$1.51, 0.78, 0.84, $-$2.36] [$-$1.51, 0.78, $-$2.36] [2,0,1]
[1,1,1] $+$[$-0.33,-$1.51, 0.01, 0.84, $-$3.34] [$-$1.51, 0.01, $-$3.34] [2,0,1]
#PermutationSumFinal Memory State[M2, M3, M5]Signature
[$-1,-1,-$1] $-$[$-0.33,-$1.51, 2.72, 0.84, $-$0.80] [$-$1.51, 2.72, $-$0.80] [0,2,1]
[$-1,-$1,1] $-$[$-0.33,-$1.51, 1.73, 0.84, $-$1.46] [$-$1.51, 1.73, $-$1.46] [0,2,1]
[$-1,1,-$1] $-$[$-0.33,-$1.51, 1.73, 0.84, $-$1.46] [$-$1.51, 1.73, $-$1.46] [0,2,1]
[1,$-1,-$1] $-$[$-0.33,-$1.51, 1.73, 0.84, $-$1.46] [$-$1.51, 1.73, $-$1.46] [0,2,1]
[$-$1,1,1] $+$[$-0.33,-$1.51, 0.78, 0.84, $-$2.36] [$-$1.51, 0.78, $-$2.36] [2,0,1]
[1,$-$1,1] $+$[$-0.33,-$1.51, 0.78, 0.84, $-$2.36] [$-$1.51, 0.78, $-$2.36] [2,0,1]
[1,1,$-$1] $+$[$-0.33,-$1.51, 0.78, 0.84, $-$2.36] [$-$1.51, 0.78, $-$2.36] [2,0,1]
[1,1,1] $+$[$-0.33,-$1.51, 0.01, 0.84, $-$3.34] [$-$1.51, 0.01, $-$3.34] [2,0,1]
Close Modal