Statistical Mechanics of Neural Networks

This book highlights a comprehensive introduction to the fundamental statistical mechanics underneath the inner workings of neural networks. The book discusses in details important concepts and techniques including the cavity method, the mean-field theory, replica techniques, the Nishimori condition, variational methods, the dynamical mean-field theory, unsupervised learning, associative memory models, perceptron models, the chaos theory of recurrent neural networks, and eigen-spectrums of neural networks, walking new learners through the theories and must-have skillsets to understand and use neural networks. The book focuses on quantitative frameworks of neural network models where the underlying mechanisms can be precisely isolated by physics of mathematical beauty and theoretical predictions. It is a good reference for students, researchers, and practitioners in the area of neural networks.

Presents major theoretical tools for the analysis of neural networks Provides concrete examples for the use of the theories in neural networks Bridges old tools and frontiers in the theoretical development of neural networks

Autorentext
Haiping Huang

Dr. Haiping Huang received his Ph.D. degree in theoretical physics from the Institute of Theoretical Physics, the Chinese Academy of Sciences. He works as an associate professor at the School of Physics, Sun Yat-sen University, China. His research interests include the origin of the computational hardness of the binary perceptron model, the theory of dimension reduction in deep neural networks, and inherent symmetry breaking in unsupervised learning. In 2021, he was awarded Excellent Young Scientists Fund by National Natural Science Foundation of China.

Inhalt
Introduction.- Spin glass models and cavity method.- Variational mean-eld theory and belief propagation.- Monte Carlo simulation methods.- High-temperature expansion.- Nishimori line.- Random energy model.- Statistical mechanical theory of Hopeld model.- Replica symmetry and replica symmetry breaking.- Statistical mechanics of restricted Boltzmann machine.- Simplest model of unsupervised learning with binary synapses.- Inherent-symmetry breaking in unsupervised learning.- Mean-eld theory of Ising Perceptron.- Mean-eld model of multi-layered Perceptron.- Mean-eld theory of dimension reduction.- Chaos theory of random recurrent neural networks.- Statistical mechanics of random matrices.- Perspectives.