In this article we study the effect of dynamically modifying the weight matrix on the performance of a neural associative memory. The dynamic modification is implemented by adding, at each step, the outer product of the current state, scaled by a suitable constant η, to the correlation weight matrix. For single-shot synchronous dynamics, we analytically obtain the optimal value of η. Although knowledge of the noise percentage is required for calculating the optimal value of η, a fairly good choice of η can be made even when the amount of noise is not known. Experimental results are provided in support of the analysis. The efficacy of the proposed modification is also experimentally verified for the case of asynchronous updating with transient length > 1.