Skip Nav Destination
Close Modal
Update search
NARROW
Format
Journal
TocHeadingTitle
Date
Availability
1-2 of 2
Asja Fischer
Close
Follow your search
Access your saved searches in your account
Would you like to receive an alert when new items match your search?
Sort by
Journal Articles
Publisher: Journals Gateway
Neural Computation (2017) 29 (3): 555–577.
Published: 01 March 2017
FIGURES
| View All (12)
Abstract
View articletitled, STDP-Compatible Approximation of Backpropagation in an Energy-Based Model
View
PDF
for article titled, STDP-Compatible Approximation of Backpropagation in an Energy-Based Model
We show that Langevin Markov chain Monte Carlo inference in an energy-based model with latent variables has the property that the early steps of inference, starting from a stationary point, correspond to propagating error gradients into internal layers, similar to backpropagation. The backpropagated error is with respect to output units that have received an outside driving force pushing them away from the stationary point. Backpropagated error gradients correspond to temporal derivatives with respect to the activation of hidden units. These lead to a weight update proportional to the product of the presynaptic firing rate and the temporal rate of change of the postsynaptic firing rate. Simulations and a theoretical argument suggest that this rate-based update rule is consistent with those associated with spike-timing-dependent plasticity. The ideas presented in this article could be an element of a theory for explaining how brains perform credit assignment in deep hierarchies as efficiently as backpropagation does, with neural computation corresponding to both approximate inference in continuous-valued latent variables and error backpropagation, at the same time.
Journal Articles
Publisher: Journals Gateway
Neural Computation (2011) 23 (3): 664–673.
Published: 01 March 2011
FIGURES
Abstract
View articletitled, Bounding the Bias of Contrastive Divergence Learning
View
PDF
for article titled, Bounding the Bias of Contrastive Divergence Learning
Optimization based on k -step contrastive divergence (CD) has become a common way to train restricted Boltzmann machines (RBMs). The k -step CD is a biased estimator of the log-likelihood gradient relying on Gibbs sampling. We derive a new upper bound for this bias. Its magnitude depends on k , the number of variables in the RBM, and the maximum change in energy that can be produced by changing a single variable. The last reflects the dependence on the absolute values of the RBM parameters. The magnitude of the bias is also affected by the distance in variation between the modeled distribution and the starting distribution of the Gibbs chain.