A Closer Look at Memorization in Deep Networks

Abstract

We examine the role of memorization in deep learning, drawing connections to capacity, generalization, and adversarial robustness. While deep networks are capable of memorizing noise data, our results suggest that they tend to prioritize learning simple patterns first. In our experiments, we expose qualitative differences in gradient-based optimization of deep neural networks (DNNs) on noise vs. real data. We also demonstrate that for appropriately tuned explicit regularization (e.g., dropout) we can degrade DNN training performance on noise datasets without compromising generalization on real data. Our analysis suggests that the notions of effective capacity which are dataset independent are unlikely to explain the generalization performance of deep networks when trained with gradient based methods because training data itself plays an important role in determining the degree of memorization.

arXiv commentAppears in Proceedings of the 34th International Conference on Machine Learning (ICML 2017), Devansh Arpit, Stanisław Jastrzębski, Nicolas Ballas, and David Krueger contributed equally to this work

References in corpus (10)

• Explaining and Harnessing Adversarial Examples
• Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
• On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
• Gradient-based Hyperparameter Optimization through Reversible Learning
• Train faster, generalize better: Stability of stochastic gradient descent
• Exponential expressivity in deep neural networks through transient chaos
• Entropy-SGD: Biasing Gradient Descent Into Wide Valleys
• On the Expressive Power of Deep Neural Networks
• An empirical analysis of the optimization of deep network loss surfaces

Cited by in corpus (214)

• mixup: Beyond Empirical Risk Minimization
• Shortcut Learning in Deep Neural Networks
• Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels
• The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
• Adaptive activation functions accelerate convergence in deep and physics-informed neural networks
• A trans-disciplinary review of deep learning research for water resources scientists
• Machine Learning for Integrating Data in Biology and Medicine: Principles, Practice, and Opportunities
• Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks
• MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels
• Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels
• Deep Learning Scaling is Predictable, Empirically
• Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting
• Image Classification with Deep Learning in the Presence of Noisy Labels: A Survey
• Annotation-efficient deep learning for automatic medical image segmentation
• Denoising Implicit Feedback for Recommendation
• Early-Learning Regularization Prevents Memorization of Noisy Labels
• Dimensionality-Driven Learning with Noisy Labels
• Generalization in Deep Learning
• Learning Representations for Neural Network-Based Classification Using the Information Bottleneck Principle
• Masking: A New Perspective of Noisy Supervision
• Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
• How does Disagreement Help Generalization against Label Corruption?
• Symmetric Cross Entropy for Robust Learning with Noisy Labels
• Neural Additive Models: Interpretable Machine Learning with Neural Nets
• Robust Federated Learning with Noisy Labels
• Curriculum Loss: Robust Learning and Generalization against Label Corruption
• Deep learning generalizes because the parameter-function map is biased towards simple functions
• Active label cleaning for improved dataset quality under resource constraints
• Overfitting Mechanism and Avoidance in Deep Neural Networks
• Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learning
• Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks
• Privacy in Deep Learning: A Survey
• Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labels
• The no-free-lunch theorems of supervised learning
• Self-Adaptive Training: beyond Empirical Risk Minimization
• The Pitfalls of Simplicity Bias in Neural Networks
• Improving the Generalization of Adversarial Training with Domain Adaptation
• Convolutional neural network identification of galaxy post-mergers in UNIONS using IllustrisTNG
• Stronger generalization bounds for deep nets via a compression approach
• A Modern Take on the Bias-Variance Tradeoff in Neural Networks
• Overparameterized Neural Networks Implement Associative Memory
• Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations
• Do We Need Zero Training Loss After Achieving Zero Training Error?
• Learning from Failure: Training Debiased Classifier from Biased Classifier
• Generalization in Deep Networks: The Role of Distance from Initialization
• Are all negatives created equal in contrastive instance discrimination?
• Understanding and Improving Early Stopping for Learning with Noisy Labels
• Stiffness: A New Perspective on Generalization in Neural Networks
• Understanding the Failure Modes of Out-of-Distribution Generalization
• An analytic theory of generalization dynamics and transfer learning in deep linear networks
• How to Start Training: The Effect of Initialization and Architecture
• Training behavior of deep neural network in frequency domain
• Identity Crisis: Memorization and Generalization under Extreme Overparameterization
• Wasserstein Distances for Stereo Disparity Estimation
• Identifying Mislabeled Data using the Area Under the Margin Ranking
• IMAE for Noise-Robust Learning: Mean Absolute Error Does Not Treat Examples Equally and Gradient Magnitude's Variance Matters
• High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks
• Gradient Starvation: A Learning Proclivity in Neural Networks
• Label Noise Types and Their Effects on Deep Learning
• FINE Samples for Learning with Noisy Labels
• Understanding training and generalization in deep learning by Fourier analysis
• Unsupervised Domain Adaptation of Black-Box Source Models
• Progressive Identification of True Labels for Partial-Label Learning
• The worst of both worlds: A comparative analysis of errors in learning from data in psychology and machine learning
• Confidence Scores Make Instance-dependent Label-noise Learning Possible
• PatchUp: A Feature-Space Block-Level Regularization Technique for Convolutional Neural Networks
• Learning from Noisy Labels for Entity-Centric Information Extraction
• Instance-dependent Label-noise Learning under a Structural Causal Model
• Explicitizing an Implicit Bias of the Frequency Principle in Two-layer Neural Networks
• Open-set Label Noise Can Improve Robustness Against Inherent Label Noise
• Supervised Domain Adaptation: A Graph Embedding Perspective and a Rectified Experimental Protocol
• Regularization via Structural Label Smoothing
• What Do Neural Networks Learn When Trained With Random Labels?
• A Diffusion Theory For Deep Learning Dynamics: Stochastic Gradient Descent Exponentially Favors Flat Minima
• Training Shallow and Thin Networks for Acceleration via Knowledge Distillation with Conditional Adversarial Networks
• Provably End-to-end Label-Noise Learning without Anchor Points
• Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularization
• Deep Frank-Wolfe For Neural Network Optimization
• Sill-Net: Feature Augmentation with Separated Illumination Representation
• Constrained Learning with Non-Convex Losses
• Heteroskedastic and Imbalanced Deep Learning with Adaptive Regularization
• Knowledge Consistency between Neural Networks and Beyond
• MetaLabelNet: Learning to Generate Soft-Labels from Noisy-Labels
• Normalized Direction-preserving Adam
• Assessing Generalization of SGD via Disagreement
• Memorization in Overparameterized Autoencoders
• How does Early Stopping Help Generalization against Label Noise?
• Visualizing the PHATE of Neural Networks
• Flow-Mixup: Classifying Multi-labeled Medical Images with Corrupted Labels
• Discovering and Explaining the Representation Bottleneck of DNNs
• ProtoAttend: Attention-Based Prototypical Learning
• Source-free Domain Adaptation via Avatar Prototype Generation and Adaptation
• An LSTM approach to Forecast Migration using Google Trends
• Understanding the Interaction of Adversarial Training with Noisy Labels
• Ensemble Robustness and Generalization of Stochastic Deep Learning Algorithms
• Exploring Memorization in Adversarial Training
• What they do when in doubt: a study of inductive biases in seq2seq learners
• ProSelfLC: Progressive Self Label Correction for Training Robust Deep Neural Networks
• The Benefits of Over-parameterization at Initialization in Deep ReLU Networks
• Generalization in Machine Learning via Analytical Learning Theory
• DNN or k-NN: That is the Generalize vs. Memorize Question
• Frequency Principle in Deep Learning with General Loss Functions and Its Potential Application
• The Low-Rank Simplicity Bias in Deep Networks
• SemiNLL: A Framework of Noisy-Label Learning by Semi-Supervised Learning
• Affect Expression Behaviour Analysis in the Wild using Consensual Collaborative Training
• Combating Noise: Semi-supervised Learning by Region Uncertainty Quantification
• A Capacity Scaling Law for Artificial Neural Networks
• Split Batch Normalization: Improving Semi-Supervised Learning under Domain Shift
• Searching to Exploit Memorization Effect in Learning from Corrupted Labels
• Augmentation Strategies for Learning with Noisy Labels
• Consensual Collaborative Training And Knowledge Distillation Based Facial Expression Recognition Under Noisy Annotations
• On Mixup Regularization
• Derivative Manipulation for General Example Weighting
• Needles in Haystacks: On Classifying Tiny Objects in Large Images
• Is SGD a Bayesian sampler? Well, almost
• Meta-learners' learning dynamics are unlike learners'
• Towards Understanding the Condensation of Neural Networks at Initial Training
• Robustness study of noisy annotation in deep learning based medical image segmentation
• Targeted Nonlinear Adversarial Perturbations in Images and Videos
• Federated Noisy Client Learning
• Robust Training with Ensemble Consensus
• Convolutional Normalization: Improving Deep Convolutional Network Robustness and Training
• Wasserstein Adversarial Regularization (WAR) on label noise
• Fairness Testing of Deep Image Classification with Adequacy Metrics
• Instance Correction for Learning with Open-set Noisy Labels
• Beyond Error Propagation in Neural Machine Translation: Characteristics of Language Also Matter
• Butterfly: One-step Approach towards Wildly Unsupervised Domain Adaptation
• Empirical Study of Easy and Hard Examples in CNN Training
• The shape and simplicity biases of adversarially robust ImageNet-trained CNNs
• Artificial Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting
• Label-Noise Robust Generative Adversarial Networks
• A Closer Look at Data Bias in Neural Extractive Summarization Models
• On the Effectiveness of Regularization Against Membership Inference Attacks
• DeepObliviate: A Powerful Charm for Erasing Data Residual Memory in Deep Neural Networks
• Generalization of Reinforcement Learning with Policy-Aware Adversarial Data Augmentation
• Adaptive Sample Selection for Robust Learning under Label Noise
• Self-Adaptive Training: Bridging Supervised and Self-Supervised Learning
• Improving Generalization by Controlling Label-Noise Information in Neural Network Weights
• Co-training for On-board Deep Object Detection
• Providing theoretical learning guarantees to Deep Learning Networks
• Adaptive Early-Learning Correction for Segmentation from Noisy Annotations
• Learning from Noisy Anchors for One-stage Object Detection
• Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study
• On the Diachronic Stability of Irregularity in Inflectional Morphology
• Let's Agree to Agree: Neural Networks Share Classification Order on Real Datasets
• A Unified Plug-and-Play Framework for Effective Data Denoising and Robust Abstention
• A Very Brief Introduction to Machine Learning With Applications to Communication Systems
• A Meta Approach to Defend Noisy Labels by the Manifold Regularizer PSDR
• Deep frequency principle towards understanding why deeper learning is faster
• Robust and On-the-fly Dataset Denoising for Image Classification
• Weak and Strong Gradient Directions: Explaining Memorization, Generalization, and Hardness of Examples at Scale
• Collaborative Label Correction via Entropy Thresholding
• On the Inductive Bias of Word-Character-Level Multi-Task Learning for Speech Recognition
• Data Quality Matters For Adversarial Training: An Empirical Study
• VILD: Variational Imitation Learning with Diverse-quality Demonstrations
• Deep Learning-Based Strategy for Macromolecules Classification with Imbalanced Data from Cellular Electron Cryotomography
• Learning time-scales in two-layers neural networks
• MetaInfoNet: Learning Task-Guided Information for Sample Reweighting
• Deep Bilevel Learning
• Investigating performance of neural networks and gradient boosting models approximating microscopic traffic simulations in traffic optimization tasks
• Deep Learning from Small Amount of Medical Data with Noisy Labels: A Meta-Learning Approach
• Multi-label Iterated Learning for Image Classification with Label Ambiguity
• Safeguarded Dynamic Label Regression for Generalized Noisy Supervision
• Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrection
• Distinguishing rule- and exemplar-based generalization in learning systems
• Noisy Annotation Refinement for Object Detection
• Class-Distinct and Class-Mutual Image Generation with GANs
• Detecting Memorization in ReLU Networks
• Robust Deep Learning with Active Noise Cancellation for Spatial Computing
• No Regret Sample Selection with Noisy Labels
• Supervised Domain Adaptation using Graph Embedding
• Hyperplane Arrangements of Trained ConvNets Are Biased
• A Simple yet Effective Baseline for Robust Deep Learning with Noisy Labels
• Why flatness does and does not correlate with generalization for deep neural networks
• Spectral Roll-off Points Variations: Exploring Useful Information in Feature Maps by Its Variations
• Learning to Combat Noisy Labels via Classification Margins
• On Robustness and Bias Analysis of BERT-based Relation Extraction
• Limited Gradient Descent: Learning With Noisy Labels
• Robust Membership Encoding: Inference Attacks and Copyright Protection for Deep Learning
• Rethink the Connections among Generalization, Memorization and the Spectral Bias of DNNs
• Dynamic Sparsity Neural Networks for Automatic Speech Recognition
• Follow Your Nose -- Which Code Smells are Worth Chasing?
• Investigating the Compositional Structure Of Deep Neural Networks
• Impact of Training Dataset Size on Neural Answer Selection Models
• Trap of Feature Diversity in the Learning of MLPs
• Intraclass clustering: an implicit learning ability that regularizes DNNs
• Probably Approximately Correct Constrained Learning
• On the Orthogonality of Knowledge Distillation with Other Techniques: From an Ensemble Perspective
• Embedding Principle of Loss Landscape of Deep Neural Networks
• Switched linear projections for neural network interpretability
• Network Cooperation with Progressive Disambiguation for Partial Label Learning
• Deep Networks as Logical Circuits: Generalization and Interpretation
• Phases of learning dynamics in artificial neural networks: with or without mislabeled data
• Topological Data Analysis of Decision Boundaries with Application to Model Selection
• Halo: Learning Semantics-Aware Representations for Cross-Lingual Information Extraction
• Noisy-Labeled NER with Confidence Estimation
• Matrix Smoothing: A Regularization for DNN with Transition Matrix under Noisy Labels
• Memory and attention in deep learning
• ComSum: Commit Messages Summarization and Meaning Preservation
• Learning to Rectify for Robust Learning with Noisy Labels
• Investigation of Training Label Error Impact on RNN-T
• Neural Networks and Denotation
• Batch Inverse-Variance Weighting: Deep Heteroscedastic Regression
• Semi-Supervised Domain Generalization with Evolving Intermediate Domain
• Efficient Classification of Very Large Images with Tiny Objects
• Using Under-trained Deep Ensembles to Learn Under Extreme Label Noise
• An investigation of model-free planning
• Detecting Learning vs Memorization in Deep Neural Networks using Shared Structure Validation Sets
• Geometry-Aware Self-Training for Unsupervised Domain Adaptationon Object Point Clouds
• Stop memorizing: A data-dependent regularization framework for intrinsic pattern learning
• Improving Generalization of Deep Fault Detection Models in the Presence of Mislabeled Data
• Learning the gravitational force law and other analytic functions
• A Theoretical Analysis of Learning with Noisily Labeled Data
• Embedding Principle: a hierarchical structure of loss landscape of deep neural networks