Abstract
We improve recently published results about resources of restricted Boltzmann machines (RBM) and deep belief networks (DBN) required to make them universal approximators. We show that any distribution on the set
of binary vectors of length
can be arbitrarily well approximated by an RBM with
hidden units, where
is the minimal number of pairs of binary vectors differing in only one entry such that their union contains the support set of
. In important cases this number is half the cardinality of the support set of
(given in Le Roux & Bengio, 2008). We construct a DBN with
, hidden layers of width
that is capable of approximating any distribution on
arbitrarily well. This confirms a conjecture presented in Le Roux and Bengio (2010).
Issue Section:
Letters
© 2011 Massachusetts Institute of Technology
2011
You do not currently have access to this content.