Artificial neural networks (ANNs), while exceptionally useful for classification, are vulnerable to misdirection. Small amounts of noise can significantly affect their ability to correctly complete a task. Instead of generalizing concepts, ANNs seem to focus on surface statistical regularities in a given task. Here we compare how recurrent artificial neural networks, long short-term memory units, and Markov Brains sense and remember their environments. We show that information in Markov Brains is localized and sparsely distributed, while the other neural network substrates “smear” information about the environment across all nodes, which makes them vulnerable to noise.

Pointwise partial information decomposition provides a means to quantify information modification in discrete systems exhibiting intrinsic distributed computation. In his seminal “Computation at the Edge of Chaos”, Chris Langton investigated how intrinsic computation emerges in cellular automata which support the three primitive functions of computation—information storage, transfer, and modification. Despite the appealing description, Langton gave no precise information-theoretic definition of the three primitive functions. In the decades since, information storage and transfer have been defined; however, a satisfactory definition of information modification has proven to be more elusive. This paper uses the recently introduced pointwise partial information decomposition to provide a quantitative measure of information modification. Moreover, this approach provides a hierarchy of different types of modifications, which each combine or synthesis different combinations of stored or transferred information. This ability to identify different types of information modification events in both space and time is exemplified with an application to cellular automata.