Skip to Main Content

Advertisement

MIT Press Direct, home Open Menu
header search
Search
Advanced Search
Skip Nav Destination

Singularities Affect Dynamics of Learning in Neuromanifolds

Neural Computation (2006) 18 (5): 1007–1065.
This article has been cited by the following articles in journals that are participating in Crossref Cited-by Linking.
Stochastic collapse: how gradient noise attracts SGD dynamics towards simpler subnetworks*
  • Feng Chen
  • Daniel Kunin
  • Atsushi Yamamura (山村篤志)
  • Surya Ganguli
Journal of Statistical Mechanics: Theory and Experiment (2024) 2024 (10): 104023.
Token singularity understanding and removal for transformers
  • Dan Wang
  • Licheng Jiao
  • Ruohan Zhang
  • Shuyuan Yang
  • Fang Liu
Knowledge-Based Systems (2024) 293: 111718.
Applications of information geometry to spiking neural network activity
  • Jacob T. Crosser
  • Braden A. W. Brinkman
Physical Review E (2024) 109 (2)
Second-Order Structure Optimization of Fully Complex-Valued Neural Networks
  • Zhidong Wang
  • He Huang
IEEE Transactions on Emerging Topics in Computational Intelligence (2024) 8 (3): 2349.
Information geometry of dynamics on graphs and hypergraphs
  • Tetsuya J. Kobayashi
  • Dimitri Loutchko
  • Atsushi Kamimura
  • Shuhei A. Horiguchi
  • Yuki Sughiyama
Information Geometry (2024) 7 (1): 97.
DESCINet: A hierarchical deep convolutional neural network with skip connection for long time series forecasting
  • André Quintiliano Bezerra Silva
  • Wesley Nunes Gonçalves
  • Edson Takashi Matsubara
Expert Systems with Applications (2023) 228: 120246.
Neural Field Continuum Limits and the Structure–Function Partitioning of Cognitive–Emotional Brain Networks
  • Kevin B. Clark
Biology (2023) 12 (3): 352.
Fisher Information Matrix and its Application of Bipolar Activation Function Based Multilayer Perceptrons With General Gaussian Input
  • Weili Guo
  • Guangyu Li
  • Jianfeng Lu
IEEE Access (2022) 10: 128250.
Wind power and load temporal dependence model based on dynamic Bayesian network
  • Hongtao Wang
  • Bin Zou
Electrical Engineering (2022) 104 (3): 1265.
Learning dynamics of kernel-based deep neural networks in manifolds
  • Wei Wu
  • Xiaoyuan Jing
  • Wencai Du
  • Guoliang Chen
Science China Information Sciences (2021) 64 (11)
The Loss Surface Of Deep Linear Networks Viewed Through The Algebraic Geometry Lens
  • Dhagash Mehta
  • Tianran Chen
  • Tingting Tang
  • Jonathan Hauenstein
IEEE Transactions on Pattern Analysis and Machine Intelligence 1.
Learning dynamics of gradient descent optimization in deep neural networks
  • Wei Wu
  • Xiaoyuan Jing
  • Wencai Du
  • Guoliang Chen
Science China Information Sciences (2021) 64 (5)
Singular Learning of Deep Multilayer Perceptrons for EEG-Based Emotion Recognition
  • Weili Guo
  • Guangyu Li
  • Jianfeng Lu
  • Jian Yang
Frontiers in Computer Science (2021) 3
Adaptive Natural Gradient Method for Learning of Stochastic Neural Networks in Mini-Batch Mode
  • Hyeyoung Park
  • Kwanyong Lee
Applied Sciences (2019) 9 (21): 4568.
SRNET: A Shallow Skip Connection Based Convolutional Neural Network Design for Resolving Singularities
  • Robail Yasrab
Journal of Computer Science and Technology (2019) 34 (4): 924.
Statistical mechanical analysis of learning dynamics of two-layer perceptron with multiple output units
  • Yuki Yoshida
  • Ryo Karakida
  • Masato Okada
  • Shun-Ichi Amari
Journal of Physics A: Mathematical and Theoretical (2019) 52 (18): 184002.
Fisher Information Matrix of Unipolar Activation Function-Based Multilayer Perceptrons
  • Weili Guo
  • Yew-Soon Ong
  • Yingjiang Zhou
  • Jaime Rubio Hervas
  • Aiguo Song
  • Haikun Wei
IEEE Transactions on Cybernetics (2019) 49 (8): 3088.
Influence Area of Overlap Singularity in Multilayer Perceptrons
  • Weili Guo
  • Yuan Yang
  • Yingjiang Zhou
  • Yushun Tan
  • Haikun Wei
  • Aiguo Song
  • Guochen Pang
IEEE Access (2018) 6: 60214.
Stability analysis of opposite singularity in multilayer perceptrons
  • Weili Guo
  • Junsheng Zhao
  • Jinxia Zhang
  • Haikun Wei
  • Aiguo Song
  • Kanjian Zhang
Neurocomputing (2018) 282: 192.
Redundancy in synaptic connections enables neurons to learn optimally
  • Naoki Hiratani
  • Tomoki Fukai
Proceedings of the National Academy of Sciences (2018) 115 (29)
Resolution of singularities via deep complex‐valued neural networks
  • Tohru Nitta
Mathematical Methods in the Applied Sciences (2018) 41 (11): 4170.
Dynamics of Learning in MLP: Natural Gradient and Singularity Revisited
  • Shun-ichi Amari
  • Tomoko Ozeki
  • Ryo Karakida
  • Yuki Yoshida
  • Masato Okada
Neural Computation (2018) 30 (1): 1.
Sparse Bayesian linear regression with latent masking variables
  • Yohei Kondo
  • Kohei Hayashi
  • Shin-ichi Maeda
Neurocomputing (2017) 258: 3.
Parameter Identifiability in Statistical Machine Learning: A Review
  • Zhi-Yong Ran
  • Bao-Gang Hu
Neural Computation (2017) 29 (5): 1151.
Resolution of Singularities Introduced by Hierarchical Structure in Deep Neural Networks
  • Tohru Nitta
IEEE Transactions on Neural Networks and Learning Systems (2017) 28 (10): 2282.
Reply to “Reply to ‘Determining structural identifiability of parameter learning machines’”
  • Zhi-Yong Ran
  • Bao-Gang Hu
Neurocomputing (2016) 218: 318.
Asymptotics for Regression Models Under Loss of Identifiability
  • Joseph Rynkiewicz
Sankhya A (2016) 78 (2): 155.
Learning Dynamics of a Single Polar Variable Complex-Valued Neuron
  • Tohru Nitta
Neural Computation (2015) 27 (5): 1120.
Natural Gradient Learning Algorithms for RBF Networks
  • Junsheng Zhao
  • Haikun Wei
  • Chi Zhang
  • Weiling Li
  • Weili Guo
  • Kanjian Zhang
Neural Computation (2015) 27 (2): 481.
Insight and analysis problem solving in microbes to machines
  • Kevin B. Clark
Progress in Biophysics and Molecular Biology (2015) 119 (2): 183.
Theoretical and numerical analysis of learning dynamics near singularity in multilayer perceptrons
  • Weili Guo
  • Haikun Wei
  • Junsheng Zhao
  • Kanjian Zhang
Neurocomputing (2015) 151: 390.
An identifying function approach for determining parameter structure of statistical learning machines
  • Zhi-Yong Ran
  • Bao-Gang Hu
Neurocomputing (2015) 162: 209.
Learning Dynamics of the Complex-Valued Neural Network in the Neighborhood of Singular Points
  • Tohru Nitta
Journal of Computer and Communications (2014) 02 (01): 27.
Singularities in the identification of dynamic systems
  • Junsheng Zhao
  • Haikun Wei
  • Weili Guo
  • Kanjian Zhang
Neurocomputing (2014) 140: 339.
Averaged learning equations of error-function-based multilayer perceptrons
  • Weili Guo
  • Haikun Wei
  • Junsheng Zhao
  • Kanjian Zhang
Neural Computing and Applications (2014) 25 (3-4): 825.
Determining structural identifiability of parameter learning machines
  • Zhi-Yong Ran
  • Bao-Gang Hu
Neurocomputing (2014) 127: 88.
Determining parameter identifiability from the optimization theory framework: A Kullback–Leibler divergence approach
  • Zhi-Yong Ran
  • Bao-Gang Hu
Neurocomputing (2014) 142: 307.
Gabor feature-based face recognition on product gamma manifold via region weighting
  • Yue Zhang
  • Chuancai Liu
Neurocomputing (2013) 117: 1.
Neural Networks with Comparatively Few Critical Points
  • Tohru Nitta
Procedia Computer Science (2013) 22: 269.
Dreaming of mathematical neuroscience for half a century
  • Shun-ichi Amari
Neural Networks (2013) 37: 48.
General bound of overfitting for MLP regression models
  • J. Rynkiewicz
Neurocomputing (2012) 90: 106.
Geometry of nonlinear least squares with applications to sloppy models and optimization
  • Mark K. Transtrum
  • Benjamin B. Machta
  • James P. Sethna
Physical Review E (2011) 83 (3)
Conditional Mixture Model for Correlated Neuronal Spikes
  • Shun-ichi Amari
Neural Computation (2010) 22 (7): 1718.
Equations of states in singular statistical estimation
  • Sumio Watanabe
Neural Networks (2010) 23 (1): 20.
Analysis of artificial neural network learning near temporary minima: A fuzzy logic approach
  • Ido Roth
  • Michael Margaliot
Fuzzy Sets and Systems (2010) 161 (19): 2569.
Equations of States in Statistical Learning for an Unrealizable and Regular Case
  • Sumio WATANABE
IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences (2010) E93-A (3): 617.
Singularity and Slow Convergence of the EM algorithm for Gaussian Mixtures
  • Hyeyoung Park
  • Tomoko Ozeki
Neural Processing Letters (2009) 29 (1): 45.
Dynamics of Learning in Multilayer Perceptrons Near Singularities
  • F. Cousseau
  • T. Ozeki
  • S.-i. Amari
IEEE Transactions on Neural Networks (2008) 19 (8): 1313.
THE UNIQUENESS THEOREM FOR COMPLEX-VALUED NEURAL NETWORKS WITH THRESHOLD PARAMETERS AND THE REDUNDANCY OF THE PARAMETERS
  • TOHRU NITTA
International Journal of Neural Systems (2008) 18 (02): 123.
Dynamics of learning near singularities in radial basis function networks
  • Haikun Wei
  • Shun-ichi Amari
Neural Networks (2008) 21 (7): 989.

Advertisement

  • Online ISSN 1530-888X
  • Print ISSN 0899-7667
A product of The MIT Press
Newsletter sign up

MIT Press Direct

Information

MIT Press

Contact Us

© 2025 The MIT Press
This Feature Is Available To Subscribers Only

or Create an Account

Close Modal
Close Modal
This site uses cookies. By continuing to use our website, you are agreeing to our privacy policy. No content on this site may be used to train artificial intelligence systems without permission in writing from the MIT Press.
Accept