Machine learning with neural networks

Abstract

These are lecture notes for a course on machine learning with neural networks for scientists and engineers that I have given at Gothenburg University and Chalmers Technical University in Gothenburg, Sweden. The material is organised into three parts: Hopfield networks, supervised learning of labeled data, and learning algorithms for unlabeled data sets. Part I introduces stochastic recurrent networks: Hopfield networks and Boltzmann machines. The analysis of their learning rules sets the scene for the later parts. Part II describes supervised learning with multilayer perceptrons and convolutional neural networks. This part starts with a simple geometrical interpretation of the learning rule and leads to the recent successes of convolutional networks in object recognition, recurrent networks in language processing, and reservoir computers in time-series analysis. Part III explains what neural networks can learn about data that is not labeled. This part begins with a description of unsupervised learning techniques for clustering of data, non-linear projections, and embeddings. A section on autoencoders explains how to learn without labels using convolutional networks, and the last chapter is dedicated to reinforcement learning. The overall goal of the course is to explain the fundamental principles that allow neural networks to learn, emphasising ideas and concepts that are common to all three parts. The present version does not contain exercises (copyright owned by Cambridge University Press). The complete book is available at https://www.cambridge.org/gb/academic/subjects/physics/statistical-physics/machine-learning-neural-networks-introduction-scientists-and-engineers?format=HB.

Revised version, 241 pages

References in corpus (17)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Sequence to Sequence Learning with Neural Networks
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• On the difficulty of training Recurrent Neural Networks
• Squeeze-and-Excitation Networks
• Recent Advances in Physical Reservoir Computing: A Review
• Understanding Neural Networks Through Deep Visualization
• The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
• Generalizing from a Few Examples: A Survey on Few-Shot Learning
• The Loss Surfaces of Multilayer Networks
• Generalisation in humans and deep neural networks
• More is the Same; Phase Transitions and Mean Field Theories
• Pathwise Derivatives Beyond the Reparameterization Trick
• Expressive Power and Approximation Errors of Restricted Boltzmann Machines
• Geometry of energy landscapes and the optimizability of deep neural networks
• Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space

Cited by in corpus (10)

• Navigation of micro-swimmers in steady flow: the importance of symmetries
• Deep Learning for Optical Tweezers
• Finite-time Lyapunov exponents of deep neural networks
• Fuzzy Convolution Neural Networks for Tabular Data Classification
• Quantum Annealing for Neural Network optimization problems: a new approach via Tensor Network simulations
• Short term vs. long term: optimization of microswimmer navigation on different time horizons
• Smart navigation of a gravity-driven glider with adjustable centre-of-mass
• Improving traffic sign recognition by active search
• Active gyrotactic stability of microswimmers using hydromechanical signals
• A general framework for knowledge integration in machine learning for electromagnetic scattering using quasinormal modes