Predictive coding (PC) networks are a biologically interesting class of neural networks. Their layered hierarchy mimics the reciprocal connectivity pattern observed in the mammalian cortex, and they can be trained using local learning rules that approximate backpropagation (Bogacz, 2017 ). However, despite having feedback connections that enable information to flow down the network hierarchy, discriminative PC networks are not typically generative. Clamping the output class and running the network to equilibrium yields an input sample that usually does not resemble the training input. This letter studies this phenomenon and proposes a simple solution that promotes the generation of input samples that resemble the training inputs. Simple decay, a technique already in wide use in neural networks, pushes the PC network toward a unique minimum two-norm solution, and that unique solution provably (for linear networks) matches the training inputs. The method also vastly improves the samples generated for nonlinear networks, as we demonstrate on MNIST.

Navigation and path integration in rodents seems to involve place cells, grid cells, and theta oscillations (4–12 Hz) in the local field potential. Two main theories have been proposed to explain the neurological underpinnings of how these phenomena relate to navigation and to each other. Attractor network (AN) models revolve around the idea that local excitation and long-range inhibition connectivity can spontaneously generate grid-cell-like activity patterns. Oscillator interference (OI) models propose that spatial patterns of activity are caused by the interference patterns between neural oscillators. In rats, these oscillators have a frequency close to the theta frequency. Recent studies have shown that bats do not exhibit a theta cycle when they crawl, and yet they still have grid cells. This has been interpreted as a criticism of OI models. However, OI models do not require theta oscillations. We explain why the absence of theta oscillations does not contradict OI models and discuss how the two families of models might be distinguished experimentally.