Zipser (1991) showed that the hidden unit activity of a fully recurrent neural network model, trained on a simple memory task, matched the temporal activity patterns of memory-associated neurons in monkeys performing delayed saccade or delayed match-to-sample tasks. When noise, simulating random fluctuations in neural firing rate, is added to the unit activations of this model, the effect on the memory dynamics is to slow the rate of information loss. In this paper, we show that the dynamics of the iterated sigmoid function, with gain and bias parameters, is qualitatively very similar to the tonic response properties of Zipser's multiunit model. Analysis of the simpler system provides an explanation for the effect of noise that is missing from the description of the multiunit model.