The EM Algorithm and Information Geometry in Neural Network Learning

Hidden units play an important role in neural networks, although their activation values are unknown in many learning situations. The EM algorithm (statistical algorithm) and the em algorithm (information-geometric one) have been proposed so far in this connection, and the effectiveness of such algorithms is recognized in many areas of research. The present note points out that these two algorithms are equivalent under a certain condition, although they are different in general.