This letter presents the stability analysis for two steepest descent algorithms with momentum for quadratic functions. The corresponding local optimal parameters in Torii and Hagan ( 2002 ) and Zhang ( 2013 ) are extended to the global optimal parameters, that is, both the optimal learning rates and the optimal momentum factors are obtained simultaneously which make for the fastest convergence.

Two steepest-descent algorithms with momentum for quadratic functions are considered. For a given learning rate, the sufficient and necessary conditions for the semistability of the steepest-descent algorithms with momentum are established. Moreover, the optimal momentum factors that generally make for the fastest convergence are obtained.