How transferable are features in deep neural networks?

Abstract

Many deep neural networks trained on natural images exhibit a curious phenomenon in common: on the first layer they learn features similar to Gabor filters and color blobs. Such first-layer features appear not to be specific to a particular dataset or task, but general in that they are applicable to many datasets and tasks. Features must eventually transition from general to specific by the last layer of the network, but this transition has not been studied extensively. In this paper we experimentally quantify the generality versus specificity of neurons in each layer of a deep convolutional neural network and report a few surprising results. Transferability is negatively affected by two distinct issues: (1) the specialization of higher layer neurons to their original task at the expense of performance on the target task, which was expected, and (2) optimization difficulties related to splitting networks between co-adapted neurons, which was not expected. In an example network trained on ImageNet, we demonstrate that either of these two issues may dominate, depending on whether features are transferred from the bottom, middle, or top of the network. We also document that the transferability of features decreases as the distance between the base task and target task increases, but that transferring features even from distant tasks can be better than using random features. A final surprising result is that initializing a network with transferred features from almost any number of layers can produce a boost to generalization that lingers even after fine-tuning to the target dataset.

To appear in Advances in Neural Information Processing Systems 27 (NIPS 2014)

References in corpus (3)

• Improving neural networks by preventing co-adaptation of feature detectors
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• Visualizing and Understanding Convolutional Networks

Cited by in corpus (609)

• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• Deep learning for time series classification: a review
• Learning Transferable Features with Deep Adaptation Networks
• AutoML: A Survey of the State-of-the-Art
• LSTM Fully Convolutional Networks for Time Series Classification
• Deep learning for universal linear embeddings of nonlinear dynamics
• Deep Subdomain Adaptation Network for Image Classification
• Unsupervised Domain Adaptation with Residual Transfer Networks
• Multi-Task Learning for Dense Prediction Tasks: A Survey
• Transfusion: Understanding Transfer Learning for Medical Imaging
• Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detection
• Improving Computer-aided Detection using Convolutional Neural Networks and Random View Aggregation
• Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregation
• Deep Transfer Network with Joint Distribution Adaptation: A New Intelligent Fault Diagnosis Framework for Industry Application
• Applications of Unsupervised Deep Transfer Learning to Intelligent Fault Diagnosis: A Survey and Comparative Study
• Deep Learning for Chest X-ray Analysis: A Survey
• Model Adaptation: Unsupervised Domain Adaptation without Source Data
• Pre-Processing-Free Gear Fault Diagnosis Using Small Datasets with Deep Convolutional Neural Network-Based Transfer Learning
• Hyper-Parameter Optimization: A Review of Algorithms and Applications
• Transfer learning in hybrid classical-quantum neural networks
• Revisiting Batch Normalization For Practical Domain Adaptation
• DeepNAT: Deep Convolutional Neural Network for Segmenting Neuroanatomy
• SINet: A Scale-insensitive Convolutional Neural Network for Fast Vehicle Detection
• A Comprehensive Analysis of Deep Regression
• A Survey on Incorporating Domain Knowledge into Deep Learning for Medical Image Analysis
• Multi-source Transfer Learning with Convolutional Neural Networks for Lung Pattern Analysis
• Semantic Labeling in Very High Resolution Images via a Self-Cascaded Convolutional Neural Network
• A Survey of End-to-End Driving: Architectures and Training Methods
• Automated Pavement Crack Segmentation Using U-Net-based Convolutional Neural Network
• Tent: Fully Test-time Adaptation by Entropy Minimization
• Transfer learning for time series classification
• Apple Flower Detection using Deep Convolutional Networks
• Neural basis expansion analysis with exogenous variables: Forecasting electricity prices with NBEATSx
• Microstructural Materials Design via Deep Adversarial Learning Methodology
• Deep Learning for Generic Object Detection: A Survey
• A Comprehensive Survey on Transfer Learning
• Adaptive Aggregation Networks for Class-Incremental Learning
• Forecasting day-ahead electricity prices in Europe: the importance of considering market integration
• Revisiting Few-sample BERT Fine-tuning
• What, Where and How to Transfer in SAR Target Recognition Based on Deep CNNs
• CrackGAN: Pavement Crack Detection Using Partially Accurate Ground Truths Based on Generative Adversarial Learning
• A General Framework for Adversarial Examples with Objectives
• Realistic Evaluation of Deep Semi-Supervised Learning Algorithms
• Masked Autoencoders Are Scalable Vision Learners
• Meta-Learning with Latent Embedding Optimization
• What is being transferred in transfer learning?
• Deep Transfer Learning for Land Use and Land Cover Classification: A Comparative Study
• DarkneTZ: Towards Model Privacy at the Edge using Trusted Execution Environments
• Data-dependent Initializations of Convolutional Neural Networks
• Image Forgery Localization Based on Multi-Scale Convolutional Neural Networks
• Automatic Model Based Dataset Generation for Fast and Accurate Crop and Weeds Detection
• Towards Robust Evaluations of Continual Learning
• Hybrid CNN and Dictionary-Based Models for Scene Recognition and Domain Adaptation
• Deep Residual Correction Network for Partial Domain Adaptation
• How Does Learning Rate Decay Help Modern Neural Networks?
• Deep Learning in Photoacoustic Tomography: Current approaches and future directions
• A Survey on Data Collection for Machine Learning: a Big Data -- AI Integration Perspective
• Automated Machine Learning: State-of-The-Art and Open Challenges
• Meta-learning with differentiable closed-form solvers
• In-field high throughput grapevine phenotyping with a consumer-grade depth camera
• A Deeper Look at Facial Expression Dataset Bias
• Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation
• Learning the PE Header, Malware Detection with Minimal Domain Knowledge
• Physics-informed Neural Networks for Solving Nonlinear Diffusivity and Biot's equations
• DCAN: Deep Contour-Aware Networks for Accurate Gland Segmentation
• SelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Data
• Towards Learning a Universal Non-Semantic Representation of Speech
• Interventional Few-Shot Learning
• Generalized Radiograph Representation Learning via Cross-supervision between Images and Free-text Radiology Reports
• Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANs
• Personalizing ASR for Dysarthric and Accented Speech with Limited Data
• Message-passing neural networks for high-throughput polymer screening
• Stable a posteriori LES of 2D turbulence using convolutional neural networks: Backscattering analysis and generalization to higher Re via transfer learning
• Rethinking CNN Models for Audio Classification
• Weed Density and Distribution Estimation for Precision Agriculture using Semi-Supervised Learning
• Unsupervised Domain Adaptation with Adversarial Residual Transform Networks
• Review: Deep Learning in Electron Microscopy
• A critical analysis of self-supervision, or what we can learn from a single image
• One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers
• Current Trends in Deep Learning for Earth Observation: An Open-source Benchmark Arena for Image Classification
• One-Shot Unsupervised Cross Domain Translation
• Unsupervised Domain Adaptation via Discriminative Manifold Propagation
• Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP
• Transfer Learning using CNN for Handwritten Devanagari Character Recognition
• Locally-Supervised Deep Hybrid Model for Scene Recognition
• DELTA: DEep Learning Transfer using Feature Map with Attention for Convolutional Networks
• How to Fine-Tune BERT for Text Classification?
• Generalization and Regularization in DQN
• Shape and Margin-Aware Lung Nodule Classification in Low-dose CT Images via Soft Activation Mapping
• Adapting Auxiliary Losses Using Gradient Similarity
• Continual Learning via Neural Pruning
• Practical Blind Membership Inference Attack via Differential Comparisons
• Paraphrasing Complex Network: Network Compression via Factor Transfer
• MEDIC: A Multi-Task Learning Dataset for Disaster Image Classification
• Neurogenesis Deep Learning
• Learning to Optimize Join Queries With Deep Reinforcement Learning
• What makes instance discrimination good for transfer learning?
• Overlearning Reveals Sensitive Attributes
• POBA-GA: Perturbation Optimized Black-Box Adversarial Attacks via Genetic Algorithm
• Scattering Networks for Hybrid Representation Learning
• Solar wind prediction using deep learning
• Analyzing Human-Human Interactions: A Survey
• LEEP: A New Measure to Evaluate Transferability of Learned Representations
• Feature-Critic Networks for Heterogeneous Domain Generalization
• Building medical image classifiers with very limited data using segmentation networks
• Generalized Domain Conditioned Adaptation Network
• Analysis of Social Media Data using Multimodal Deep Learning for Disaster Response
• Learning to Continually Learn
• Domain Adaptation for Ear Recognition Using Deep Convolutional Neural Networks
• The Limitations of Deep Learning in Adversarial Settings
• UVid-Net: Enhanced Semantic Segmentation of UAV Aerial Videos by Embedding Temporal Information
• Stacking-Based Deep Neural Network: Deep Analytic Network for Pattern Classification
• Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift
• A Similarity Measure for Material Appearance
• Hurricane Forecasting: A Novel Multimodal Machine Learning Framework
• Data-driven subgrid-scale modeling of forced Burgers turbulence using deep learning with generalization to higher Reynolds numbers via transfer learning
• Segmentation of structural parts of rosebush plants with 3D point-based deep learning methods
• Rethinking the Hyperparameters for Fine-tuning
• Cycle Self-Training for Domain Adaptation
• Video Representation Learning with Visual Tempo Consistency
• Gastric histopathology image segmentation using a hierarchical conditional random field
• Solar Flare Intensity Prediction with Machine Learning Models
• Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT
• Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation
• Multi-modal Deep Analysis for Multimedia
• AutoTune: Automatically Tuning Convolutional Neural Networks for Improved Transfer Learning
• Graying the black box: Understanding DQNs
• Overcoming Small Minirhizotron Datasets Using Transfer Learning
• On Leveraging Pretrained GANs for Generation with Limited Data
• OSLNet: Deep Small-Sample Classification with an Orthogonal Softmax Layer
• A Theoretical Explanation for Perplexing Behaviors of Backpropagation-based Visualizations
• Automated Pruning for Deep Neural Network Compression
• Arabic Handwritten Character Recognition based on Convolution Neural Networks and Support Vector Machine
• Few-Shot Named Entity Recognition: A Comprehensive Study
• Hi Sigma, do I have the Coronavirus?: Call for a New Artificial Intelligence Approach to Support Health Care Professionals Dealing With The COVID-19 Pandemic
• On the Binding Problem in Artificial Neural Networks
• Speech-Driven Expressive Talking Lips with Conditional Sequential Generative Adversarial Networks
• Hermite-Gaussian Mode Detection via Convolution Neural Networks
• AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence
• Fawkes: Protecting Privacy against Unauthorized Deep Learning Models
• Combining pretrained CNN feature extractors to enhance clustering of complex natural images
• Dynamic Prediction of ICU Mortality Risk Using Domain Adaptation
• Deep Transfer Learning Methods for Colon Cancer Classification in Confocal Laser Microscopy Images
• Transfer Learning and SpecAugment applied to SSVEP Based BCI Classification
• PnP-AdaNet: Plug-and-Play Adversarial Domain Adaptation Network with a Benchmark at Cross-modality Cardiac Segmentation
• K for the Price of 1: Parameter-efficient Multi-task and Transfer Learning
• Comprehensive Instructional Video Analysis: The COIN Dataset and Performance Evaluation
• Mapping distinct phase transitions to a neural network
• Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutions
• Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis
• A Selective Overview of Deep Learning
• LogME: Practical Assessment of Pre-trained Models for Transfer Learning
• Transient-optimised real-bogus classification with Bayesian Convolutional Neural Networks -- sifting the GOTO candidate stream
• Semi-Automatic Data Annotation guided by Feature Space Projection
• Never Stop Learning: The Effectiveness of Fine-Tuning in Robotic Reinforcement Learning
• Learning from Few Samples: A Survey
• Transfer Learning for Future Wireless Networks: A Comprehensive Survey
• FixyNN: Efficient Hardware for Mobile Computer Vision via Transfer Learning
• Learning to Transfer Examples for Partial Domain Adaptation
• Boosting offline handwritten text recognition in historical documents with few labeled lines
• Exploring the Limits of Large Scale Pre-training
• 3-D Convolutional Neural Networks for Glioblastoma Segmentation
• In-domain representation learning for remote sensing
• Energy efficient distributed analytics at the edge of the network for IoT environments
• Deep Layer Aggregation
• Machine Learning Methods for Track Classification in the AT-TPC
• Neural network interpretation using descrambler groups
• Small-GAN: Speeding Up GAN Training Using Core-sets
• Multi-Task Zipping via Layer-wise Neuron Sharing
• Autoencoder Node Saliency: Selecting Relevant Latent Representations
• On the Decision Boundary of Deep Neural Networks
• Transfer Learning with Dynamic Adversarial Adaptation Network
• Cognitive simulation models for inertial confinement fusion: Combining simulation and experimental data
• Regularizing Deep Multi-Task Networks using Orthogonal Gradients
• Characterizing soot in TEM images using a convolutional neural network
• The Impact of Preprocessing on Deep Representations for Iris Recognition on Unconstrained Environments
• Drop to Adapt: Learning Discriminative Features for Unsupervised Domain Adaptation
• White Box Methods for Explanations of Convolutional Neural Networks in Image Classification Tasks
• Pose is all you need: The pose only group activity recognition system (POGARS)
• Diff-DAC: Distributed Actor-Critic for Average Multitask Deep Reinforcement Learning
• Score Function Features for Discriminative Learning: Matrix and Tensor Framework
• A Study of the Generalizability of Self-Supervised Representations
• An Ensemble of Transfer, Semi-supervised and Supervised Learning Methods for Pathological Heart Sound Classification
• Machine Learning with Confidential Computing: A Systematization of Knowledge
• What Do Neural Networks Learn When Trained With Random Labels?
• Large-Scale Visual Active Learning with Deep Probabilistic Ensembles
• Non-Structured DNN Weight Pruning -- Is It Beneficial in Any Platform?
• How Much Off-The-Shelf Knowledge Is Transferable From Natural Images To Pathology Images?
• Socially Enhanced Situation Awareness from Microblogs using Artificial Intelligence: A Survey
• Spatiotemporal Filtering for Event-Based Action Recognition
• COVID-19 Screening Using Residual Attention Network an Artificial Intelligence Approach
• Scaling Video Analytics on Constrained Edge Nodes
• Interpreting CNNs via Decision Trees
• Scalable Transfer Learning with Expert Models
• Not Just Privacy: Improving Performance of Private Deep Learning in Mobile Cloud
• RC-DARTS: Resource Constrained Differentiable Architecture Search
• Minimum Class Confusion for Versatile Domain Adaptation
• Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer
• Two Sides of the Same Coin: White-box and Black-box Attacks for Transfer Learning
• TVT: Transferable Vision Transformer for Unsupervised Domain Adaptation
• Greedy Layerwise Learning Can Scale to ImageNet
• Supervised Transfer Learning at Scale for Medical Imaging
• Harmonizing Transferability and Discriminability for Adapting Object Detectors
• Colorectal cancer diagnosis from histology images: A comparative study
• FEED: Feature-level Ensemble for Knowledge Distillation
• Factors of Transferability for a Generic ConvNet Representation
• Probing transfer learning with a model of synthetic correlated datasets
• Instance Weighted Incremental Evolution Strategies for Reinforcement Learning in Dynamic Environments
• Glyph: Fast and Accurately Training Deep Neural Networks on Encrypted Data
• Deep Learning for Scene Classification: A Survey
• LaSOT: A High-quality Benchmark for Large-scale Single Object Tracking
• Hebbian Semi-Supervised Learning in a Sample Efficiency Setting
• REMIND Your Neural Network to Prevent Catastrophic Forgetting
• Brain MRI Tumor Segmentation with Adversarial Networks
• Locality and compositionality in zero-shot learning
• Transferable Perturbations of Deep Feature Distributions
• Preconditioned training of normalizing flows for variational inference in inverse problems
• Knowledge Consistency between Neural Networks and Beyond
• DeepHealth: Review and challenges of artificial intelligence in health informatics
• Transferable Calibration with Lower Bias and Variance in Domain Adaptation
• Coupling Adversarial Learning with Selective Voting Strategy for Distribution Alignment in Partial Domain Adaptation
• Certified Robustness to Label-Flipping Attacks via Randomized Smoothing
• DSDANet: Deep Siamese Domain Adaptation Convolutional Neural Network for Cross-domain Change Detection
• Stein variational gradient descent with local approximations
• On Last-Layer Algorithms for Classification: Decoupling Representation from Uncertainty Estimation
• A Panda? No, It's a Sloth: Slowdown Attacks on Adaptive Multi-Exit Neural Network Inference
• Bridge Damage Detection using a Single-Stage Detector and Field Inspection Images
• Deep Learning for Design and Retrieval of Nano-photonic Structures
• Multi-Modal 3D Object Detection in Autonomous Driving: a Survey
• Energy landscape analysis based on the Ising model: Tutorial review
• Cross-domain Detection via Graph-induced Prototype Alignment
• General Purpose (GenP) Bioimage Ensemble of Handcrafted and Learned Features with Data Augmentation
• Modeling Spatial and Temporal Cues for Multi-label Facial Action Unit Detection
• Bridging the Theoretical Bound and Deep Algorithms for Open Set Domain Adaptation
• Fast Construction of Correcting Ensembles for Legacy Artificial Intelligence Systems: Algorithms and a Case Study
• Multitask Balanced and Recalibrated Network for Medical Code Prediction
• Language models and Automated Essay Scoring
• Hot-Starting the Ac Power Flow with Convolutional Neural Networks
• Revisiting Meta-Learning as Supervised Learning
• When Semi-Supervised Learning Meets Transfer Learning: Training Strategies, Models and Datasets
• A Broader Study of Cross-Domain Few-Shot Learning
• Virtual-to-Real-World Transfer Learning for Robots on Wilderness Trails
• A Real-time Defense against Website Fingerprinting Attacks
• Walling up Backdoors in Intrusion Detection Systems
• Characterizing and Avoiding Negative Transfer
• Faster Uncertainty Quantification for Inverse Problems with Conditional Normalizing Flows
• Recognition Of Surface Defects On Steel Sheet Using Transfer Learning
• A Flexible Selection Scheme for Minimum-Effort Transfer Learning
• RoboCup 2019 AdultSize Winner NimbRo: Deep Learning Perception, In-Walk Kick, Push Recovery, and Team Play Capabilities
• A Comprehensive Approach for UAV Small Object Detection with Simulation-based Transfer Learning and Adaptive Fusion
• A visual approach for age and gender identification on Twitter
• Associate-3Ddet: Perceptual-to-Conceptual Association for 3D Point Cloud Object Detection
• On Learning and Learned Data Representation by Capsule Networks
• Why my photos look sideways or upside down? Detecting Canonical Orientation of Images using Convolutional Neural Networks
• GNAS: A Greedy Neural Architecture Search Method for Multi-Attribute Learning
• Land Use and Land Cover Classification Using Deep Learning Techniques
• POD-DL-ROM: enhancing deep learning-based reduced order models for nonlinear parametrized PDEs by proper orthogonal decomposition
• MirBot: A collaborative object recognition system for smartphones using convolutional neural networks
• Robust Watermarking of Neural Network with Exponential Weighting
• Transferring Autonomous Driving Knowledge on Simulated and Real Intersections
• Data-driven geophysics: from dictionary learning to deep learning
• Active Image Synthesis for Efficient Labeling
• Rectified Decision Trees: Towards Interpretability, Compression and Empirical Soundness
• Robust Navigation with Language Pretraining and Stochastic Sampling
• The effect of task and training on intermediate representations in convolutional neural networks revealed with modified RV similarity analysis
• T-Miner: A Generative Approach to Defend Against Trojan Attacks on DNN-based Text Classification
• A Secure Mobile Authentication Alternative to Biometrics
• How to fine-tune deep neural networks in few-shot learning?
• Bi-tuning of Pre-trained Representations
• Hierarchically Organized Latent Modules for Exploratory Search in Morphogenetic Systems
• Frustratingly Easy Transferability Estimation
• UI-Net: Interactive Artificial Neural Networks for Iterative Image Segmentation Based on a User Model
• Flow imaging as an alternative to pressure transducers through vision transformers and convolutional neural networks
• A Comprehensive Analysis of Information Leakage in Deep Transfer Learning
• Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis
• Beyond Fine-Tuning: Transferring Behavior in Reinforcement Learning
• Enhanced Behavioral Cloning Based self-driving Car Using Transfer Learning
• Domain-specific Communication Optimization for Distributed DNN Training
• CNN Based Hashing for Image Retrieval
• Transfer learning driven design optimization for inertial confinement fusion
• Multimodal Side-Tuning for Document Classification
• Do Deep Neural Networks Forget Facial Action Units? -- Exploring the Effects of Transfer Learning in Health Related Facial Expression Recognition
• Multi-Object Classification and Unsupervised Scene Understanding Using Deep Learning Features and Latent Tree Probabilistic Models
• Deep Image Category Discovery using a Transferred Similarity Function
• Ranking and Tuning Pre-trained Models: A New Paradigm for Exploiting Model Hubs
• Gradients as Features for Deep Representation Learning
• Adversarial Alignment of Class Prediction Uncertainties for Domain Adaptation
• How Transferable are CNN-based Features for Age and Gender Classification?
• Source Separation with Deep Generative Priors
• Transfer Learning for Human Activity Recognition using Representational Analysis of Neural Networks
• Advances in Deep Learning for Hyperspectral Image Analysis--Addressing Challenges Arising in Practical Imaging Scenarios
• Toward Automatic Threat Recognition for Airport X-ray Baggage Screening with Deep Convolutional Object Detection
• IMUTube: Automatic Extraction of Virtual on-body Accelerometry from Video for Human Activity Recognition
• Adaptive Denoising via GainTuning
• Instance Credibility Inference for Few-Shot Learning
• Learning from a Complementary-label Source Domain: Theory and Algorithms
• Multi-Modal Detection of Alzheimer's Disease from Speech and Text
• Heterogeneous Data-Aware Federated Learning
• Accelerating Deep Unsupervised Domain Adaptation with Transfer Channel Pruning
• SuperTML: Two-Dimensional Word Embedding for the Precognition on Structured Tabular Data
• Retrieving Similar E-Commerce Images Using Deep Learning
• Rethinking Supervised Pre-training for Better Downstream Transferring
• Seismic data denoising and deblending using deep learning
• Assessing Car Damage using Mask R-CNN
• COALA: Co-Aligned Autoencoders for Learning Semantically Enriched Audio Representations
• Neural network augmented wave-equation simulation
• Learning to Combine: Knowledge Aggregation for Multi-Source Domain Adaptation
• Structured Pruning for Efficient ConvNets via Incremental Regularization
• Incremental Learning Using a Grow-and-Prune Paradigm with Efficient Neural Networks
• Embryo staging with weakly-supervised region selection and dynamically-decoded predictions
• FSD-10: A Dataset for Competitive Sports Content Analysis
• To Transfer or Not to Transfer: Misclassification Attacks Against Transfer Learned Text Classifiers
• Adversarial Domain Adaptation with Domain Mixup
• AdapNet: Adaptability Decomposing Encoder-Decoder Network for Weakly Supervised Action Recognition and Localization
• Transfer Learning for British Sign Language Modelling
• Towards Efficient Unconstrained Palmprint Recognition via Deep Distillation Hashing
• Aggregate or Not? Exploring Where to Privatize in DNN Based Federated Learning Under Different Non-IID Scenes
• Adaptive Consistency Regularization for Semi-Supervised Transfer Learning
• Mining Generalized Features for Detecting AI-Manipulated Fake Faces
• Contrastively Smoothed Class Alignment for Unsupervised Domain Adaptation
• Revealing Fine Structures of the Retinal Receptive Field by Deep Learning Networks
• Disentangled Representations in Neural Models
• Deep causal representation learning for unsupervised domain adaptation
• Transfer Learning with Deep CNNs for Gender Recognition and Age Estimation
• Disjoint Label Space Transfer Learning with Common Factorised Space
• Learning from Multiple Cities: A Meta-Learning Approach for Spatial-Temporal Prediction
• The shape and simplicity biases of adversarially robust ImageNet-trained CNNs
• Extraction of Complex DNN Models: Real Threat or Boogeyman?
• Headless Horseman: Adversarial Attacks on Transfer Learning Models
• Exploring the Vulnerability of Deep Neural Networks: A Study of Parameter Corruption
• Deep Adversarial Transition Learning using Cross-Grafted Generative Stacks
• SNN: Stacked Neural Networks
• Urban Rhapsody: Large-scale exploration of urban soundscapes
• Conditional Adversarial Networks for Multi-Domain Text Classification
• Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles
• CITISEN: A Deep Learning-Based Speech Signal-Processing Mobile Application
• Evaluation of taxonomic and neural embedding methods for calculating semantic similarity
• Unsupervised Transfer Learning via BERT Neuron Selection
• Composer Style Classification of Piano Sheet Music Images Using Language Model Pretraining
• Fast CRDNN: Towards on Site Training of Mobile Construction Machines
• Disentangled Cumulants Help Successor Representations Transfer to New Tasks
• Learning through structure: towards deep neuromorphic knowledge graph embeddings
• Network Transplanting
• Improving Computational Efficiency in Visual Reinforcement Learning via Stored Embeddings
• Actor Critic with Differentially Private Critic
• TransTailor: Pruning the Pre-trained Model for Improved Transfer Learning
• Geometric Scattering for Graph Data Analysis
• An Empirical Evaluation of Adversarial Robustness under Transfer Learning
• Domain Generalization for Document Authentication against Practical Recapturing Attacks
• Move-to-Data: A new Continual Learning approach with Deep CNNs, Application for image-class recognition
• Learning to Match Distributions for Domain Adaptation
• Data-Free Knowledge Amalgamation via Group-Stack Dual-GAN
• A Multi-Stage Attentive Transfer Learning Framework for Improving COVID-19 Diagnosis
• Initialization of ReLUs for Dynamical Isometry
• Classifying the classifier: dissecting the weight space of neural networks
• Differentially Private Continual Learning
• Guidelines and Benchmarks for Deployment of Deep Learning Models on Smartphones as Real-Time Apps
• Shredder: Learning Noise Distributions to Protect Inference Privacy
• Improving the Capabilities of Large Language Model Based Marketing Analytics Copilots With Semantic Search And Fine-Tuning
• Neural Dataset Generality
• Towards Neural Network Patching: Evaluating Engagement-Layers and Patch-Architectures
• "Train one, Classify one, Teach one" -- Cross-surgery transfer learning for surgical step recognition
• Non-binary deep transfer learning for image classification
• Dealing with Adversarial Player Strategies in the Neural Network Game iNNk through Ensemble Learning
• Detecting Localized Adversarial Examples: A Generic Approach using Critical Region Analysis
• BiPC: Bidirectional Probability Calibration for Unsupervised Domain Adaption
• Deep learning in bioinformatics: introduction, application, and perspective in big data era
• A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networks
• Unsupervised Deep Feature Transfer for Low Resolution Image Classification
• Generation of Realistic Cloud Access Times for Mobile Application Testing using Transfer Learning
• Deep Learning and Control Algorithms of Direct Perception for Autonomous Driving
• I3Net: Implicit Instance-Invariant Network for Adapting One-Stage Object Detectors
• Discriminative Adversarial Domain Adaptation
• Improving Robustness and Generality of NLP Models Using Disentangled Representations
• A Review of Meta-level Learning in the Context of Multi-component, Multi-level Evolving Prediction Systems
• On Evolving Attention Towards Domain Adaptation
• Neural Data Server: A Large-Scale Search Engine for Transfer Learning Data
• The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks
• Multi-Stage Transfer Learning with an Application to Selection Process
• Proof-of-Learning: Definitions and Practice
• Detecting retail products in situ using CNN without human effort labeling
• Meta-learning with negative learning rates
• Lifelong Learning using Eigentasks: Task Separation, Skill Acquisition, and Selective Transfer
• Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks
• Neural Networks Trained on Natural Scenes Exhibit Gestalt Closure
• AutoFT: Automatic Fine-Tune for Parameters Transfer Learning in Click-Through Rate Prediction
• MMFNet: A Multi-modality MRI Fusion Network for Segmentation of Nasopharyngeal Carcinoma
• User-specific Adaptive Fine-tuning for Cross-domain Recommendations
• Continual Learning with Self-Organizing Maps
• Challenges for cognitive decoding using deep learning methods
• ConvNets vs. Transformers: Whose Visual Representations are More Transferable?
• Detecting Colorized Images via Convolutional Neural Networks: Toward High Accuracy and Good Generalization
• The Impact of Reinitialization on Generalization in Convolutional Neural Networks
• CoReS: Compatible Representations via Stationarity
• Immigration Document Classification and Automated Response Generation
• WAFFLE: Watermarking in Federated Learning
• Spirit Distillation: A Model Compression Method with Multi-domain Knowledge Transfer
• Generative Hierarchical Features from Synthesizing Images
• Human-centric Transfer Learning Explanation via Knowledge Graph [Extended Abstract]
• Skin disease diagnosis with deep learning: a review
• Finding Quantum Critical Points with Neural-Network Quantum States
• Adaptation of a deep learning malignancy model from full-field digital mammography to digital breast tomosynthesis
• AdarGCN: Adaptive Aggregation GCN for Few-Shot Learning
• Application of Transfer Learning to Sign Language Recognition using an Inflated 3D Deep Convolutional Neural Network
• HufuNet: Embedding the Left Piece as Watermark and Keeping the Right Piece for Ownership Verification in Deep Neural Networks
• SimEx: Express Prediction of Inter-dataset Similarity by a Fleet of Autoencoders
• Deeply Aligned Adaptation for Cross-domain Object Detection
• HyperSTAR: Task-Aware Hyperparameters for Deep Networks
• Fast Crack Detection Using Convolutional Neural Network
• Tinkering Under the Hood: Interactive Zero-Shot Learning with Net Surgery
• An Application of CNNs to Time Sequenced One Dimensional Data in Radiation Detection
• Frosting Weights for Better Continual Training
• Data Augmentation and Transfer Learning Approaches Applied to Facial Expressions Recognition
• CAM-GAN: Continual Adaptation Modules for Generative Adversarial Networks
• Transfer learning from language models to image caption generators: Better models may not transfer better
• Interpretable CNNs for Object Classification
• Can deep learning help you find the perfect match?
• Lightweight and Robust Representation of Economic Scales from Satellite Imagery
• A Study of Transfer Learning in Music Source Separation
• Distributed Deep Transfer Learning by Basic Probability Assignment
• Dual Memory Architectures for Fast Deep Learning of Stream Data via an Online-Incremental-Transfer Strategy
• ORBIT: A Real-World Few-Shot Dataset for Teachable Object Recognition
• An AI-based Solution for Enhancing Delivery of Digital Learning for Future Teachers
• Adaptively-Accumulated Knowledge Transfer for Partial Domain Adaptation
• Attention Transfer from Web Images for Video Recognition
• Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning
• Modelling Domain Relationships for Transfer Learning on Retrieval-based Question Answering Systems in E-commerce
• GAN Memory with No Forgetting
• Using Latent Codes for Class Imbalance Problem in Unsupervised Domain Adaptation
• Rectified Decision Trees: Exploring the Landscape of Interpretable and Effective Machine Learning
• Pretraining Techniques for Sequence-to-Sequence Voice Conversion
• Revisiting Mid-Level Patterns for Cross-Domain Few-Shot Recognition
• Multi-stream 3D FCN with Multi-scale Deep Supervision for Multi-modality Isointense Infant Brain MR Image Segmentation
• Obstruction level detection of sewer videos using convolutional neural networks
• Semantic Hierarchical Priors for Intrinsic Image Decomposition
• Inter-subject Deep Transfer Learning for Motor Imagery EEG Decoding
• An Automated Approach for Timely Diagnosis and Prognosis of Coronavirus Disease
• A Multi-site Study of a Breast Density Deep Learning Model for Full-field Digital Mammography Images and Synthetic Mammography Images
• Investigating the Relationship Between Dropout Regularization and Model Complexity in Neural Networks
• Transfer Learning Using Classification Layer Features of CNN
• Exploiting Contextual Information with Deep Neural Networks
• Source-Free Adaptation to Measurement Shift via Bottom-Up Feature Restoration
• Learned Indexes for Dynamic Workloads
• LegaLMFiT: Efficient Short Legal Text Classification with LSTM Language Model Pre-Training
• Structural Regularization
• Sensitivity of Deep Convolutional Networks to Gabor Noise
• Understanding the Effects of Pre-Training for Object Detectors via Eigenspectrum
• How to trust unlabeled data? Instance Credibility Inference for Few-Shot Learning
• Image-to-Video Person Re-Identification by Reusing Cross-modal Embeddings
• Multi-Task Feature Learning for Knowledge Graph Enhanced Recommendation
• Effective Domain Knowledge Transfer with Soft Fine-tuning
• Sparse bottleneck neural networks for exploratory non-linear visualization of Patch-seq data
• Uncovering the Connections Between Adversarial Transferability and Knowledge Transferability
• Leveraging Socioeconomic Information and Deep Learning for Residential Load Pattern Prediction
• SSDL: Self-Supervised Domain Learning for Improved Face Recognition
• Stochastic Adversarial Gradient Embedding for Active Domain Adaptation
• A General Class of Transfer Learning Regression without Implementation Cost
• Automated ultrasound doppler angle estimation using deep learning
• Few-Shot Learning-Based Human Activity Recognition
• Detection and Annotation of Plant Organs from Digitized Herbarium Scans using Deep Learning
• MELINDA: A Multimodal Dataset for Biomedical Experiment Method Classification
• ShadowNet: A Secure and Efficient On-device Model Inference System for Convolutional Neural Networks
• Deep Transfer Clustering of Radio Signals
• Hierarchical nucleation in deep neural networks
• EnTrans:Leveraging Kinetic Energy Harvesting Signal for Transportation Mode Detection
• Developing efficient transfer learning strategies for robust scene recognition in mobile robotics using pre-trained convolutional neural networks
• Snowflake: Scaling GNNs to High-Dimensional Continuous Control via Parameter Freezing
• Domain Adaption for Knowledge Tracing
• Transfer Learning in 4D for Breast Cancer Diagnosis using Dynamic Contrast-Enhanced Magnetic Resonance Imaging
• Watermarking Graph Neural Networks based on Backdoor Attacks
• Ward2ICU: A Vital Signs Dataset of Inpatients from the General Ward
• Incremental Meta-Learning via Indirect Discriminant Alignment
• Learning Invariant Representations across Domains and Tasks
• Deep Transfer Learning for Infectious Disease Case Detection Using Electronic Medical Records
• Improving Skin Condition Classification with a Question Answering Model
• Deep Probabilistic Ensembles: Approximate Variational Inference through KL Regularization
• A feasibility study of deep neural networks for the recognition of banknotes regarding central bank requirements
• Accelerating Multi-Model Inference by Merging DNNs of Different Weights
• VideoDG: Generalizing Temporal Relations in Videos to Novel Domains
• Transfer Learning in General Lensless Imaging through Scattering Media
• Spirit Distillation: Precise Real-time Semantic Segmentation of Road Scenes with Insufficient Data
• Interpretable Neural Network Decoupling
• A Deep Reinforcement Learning Approach for Dynamic Contents Caching in HetNets
• A Fully Spiking Hybrid Neural Network for Energy-Efficient Object Detection
• Pre-trained Word Embeddings for Goal-conditional Transfer Learning in Reinforcement Learning
• HalalNet: A Deep Neural Network that Classifies the Halalness Slaughtered Chicken from their Images
• Disentanglement Then Reconstruction: Learning Compact Features for Unsupervised Domain Adaptation
• An Attention-Based Speaker Naming Method for Online Adaptation in Non-Fixed Scenarios
• Characterizing and Understanding the Generalization Error of Transfer Learning with Gibbs Algorithm
• Progressively Growing Generative Adversarial Networks for High Resolution Semantic Segmentation of Satellite Images
• Memory Efficient Meta-Learning with Large Images
• FGLP: A Federated Fine-Grained Location Prediction System for Mobile Users
• Practical Transferability Estimation for Image Classification Tasks
• ImageNet Pre-training also Transfers Non-Robustness
• PCNet: A Structure Similarity Enhancement Method for Multispectral and Multimodal Image Registration
• Classifying Textual Data with Pre-trained Vision Models through Transfer Learning and Data Transformations
• Transfer Learning and Curriculum Learning in Sokoban
• Feature Space Targeted Attacks by Statistic Alignment
• Fast Adaptation with Linearized Neural Networks
• Quantifying Predictive Uncertainty in Medical Image Analysis with Deep Kernel Learning
• Defending Democracy: Using Deep Learning to Identify and Prevent Misinformation
• AFAN: Augmented Feature Alignment Network for Cross-Domain Object Detection
• Satellite Image Classification with Deep Learning
• Interpreting Deep Learning Model Using Rule-based Method
• Why Do Better Loss Functions Lead to Less Transferable Features?
• X-model: Improving Data Efficiency in Deep Learning with A Minimax Model
• Exploring the potential of transfer learning for metamodels of heterogeneous material deformation
• Combining Domain-Specific Meta-Learners in the Parameter Space for Cross-Domain Few-Shot Classification
• Meta-learning Transferable Representations with a Single Target Domain
• Against Adversarial Learning: Naturally Distinguish Known and Unknown in Open Set Domain Adaptation
• DEPARA: Deep Attribution Graph for Deep Knowledge Transferability
• Transferable Visual Words: Exploiting the Semantics of Anatomical Patterns for Self-supervised Learning
• Medical Image Segmentation with Limited Supervision: A Review of Deep Network Models
• Sensing population distribution from satellite imagery via deep learning: model selection, neighboring effect, and systematic biases
• Multi-level neural networks for PDEs with uncertain parameters
• A Framework for Behavioral Biometric Authentication using Deep Metric Learning on Mobile Devices
• SERIL: Noise Adaptive Speech Enhancement using Regularization-based Incremental Learning
• Weighed Domain-Invariant Representation Learning for Cross-domain Sentiment Analysis
• Matching Embeddings for Domain Adaptation
• Self-Supervised Dynamic Networks for Covariate Shift Robustness
• Generating Adversarial Examples with an Optimized Quality
• Is this sentence valid? An Arabic Dataset for Commonsense Validation
• Sensitive Information Detection: Recursive Neural Networks for Encoding Context
• Neural Crossbreed: Neural Based Image Metamorphosis
• Fused Deep Convolutional Neural Network for Precision Diagnosis of COVID-19 Using Chest X-Ray Images
• Program synthesis performance constrained by non-linear spatial relations in Synthetic Visual Reasoning Test
• Auto-Classification of Retinal Diseases in the Limit of Sparse Data Using a Two-Streams Machine Learning Model
• Machine Learning Techniques to Detect and Characterise Whistler Radio Waves
• Parallelized Interactive Machine Learning on Autonomous Vehicles
• Understanding the Mechanism of Deep Learning Framework for Lesion Detection in Pathological Images with Breast Cancer
• High-Level Perceptual Similarity is Enabled by Learning Diverse Tasks
• Watermark retrieval from 3D printed objects via synthetic data training
• Unsupervised Domain Alignment to Mitigate Low Level Dataset Biases
• Growing a Brain: Fine-Tuning by Increasing Model Capacity
• Constrained Deep One-Class Feature Learning For Classifying Imbalanced Medical Images
• Quantifying and Maximizing the Benefits of Back-End Noise Adaption on Attention-Based Speech Recognition Models
• Model Adaption Object Detection System for Robot
• Cross Domain Image Matching in Presence of Outliers
• Semantic and Visual Similarities for Efficient Knowledge Transfer in CNN Training
• Towards Efficient and Secure Delivery of Data for Deep Learning with Privacy-Preserving
• Towards Efficient and Secure Delivery of Data for Training and Inference with Privacy-Preserving
• Granular Motor State Monitoring of Free Living Parkinson's Disease Patients via Deep Learning
• Progressive growing of self-organized hierarchical representations for exploration
• Transfer learning in large-scale ocean bottom seismic wavefield reconstruction
• Shareable Representations for Search Query Understanding
• Convolutional Neural Networks for User Identificationbased on Motion Sensors Represented as Image
• Dynamic Domain Adaptation for Efficient Inference
• Diabetic Retinopathy detection by retinal image recognizing
• A Transferable Adaptive Domain Adversarial Neural Network for Virtual Reality Augmented EMG-Based Gesture Recognition
• Apparel Recommender System based on Bilateral image shape features
• Shuffle-Then-Assemble: Learning Object-Agnostic Visual Relationship Features
• Improving speech recognition models with small samples for air traffic control systems
• 3D Object Segmentation for Shelf Bin Picking by Humanoid with Deep Learning and Occupancy Voxel Grid Map
• Train your classifier first: Cascade Neural Networks Training from upper layers to lower layers
• Partial Weight Adaptation for Robust DNN Inference
• Adjust Planning Strategies to Accommodate Reinforcement Learning Agents
• SpotPatch: Parameter-Efficient Transfer Learning for Mobile Object Detection
• Cross-Domain Latent Modulation for Variational Transfer Learning
• Classification of animal sounds in a hyperdiverse rainforest using Convolutional Neural Networks
• Assessing The Importance Of Colours For CNNs In Object Recognition
• Anomaly Detection in Time Series with Triadic Motif Fields and Application in Atrial Fibrillation ECG Classification
• Residual Continual Learning
• Fine tuning U-Net for ultrasound image segmentation: which layers?
• TAGLETS: A System for Automatic Semi-Supervised Learning with Auxiliary Data
• Convergence Analysis of Homotopy-SGD for non-convex optimization
• Refining Automatic Speech Recognition System for older adults
• Low-Cost Transfer Learning of Face Tasks
• Classification of optics-free images with deep neural networks
• Mixed Set Domain Adaptation
• FADACS: A Few-shot Adversarial Domain Adaptation Architecture for Context-Aware Parking Availability Sensing
• All-Weather Object Recognition Using Radar and Infrared Sensing
• Dataset Bias in Few-shot Image Recognition
• Efficient Modelling Across Time of Human Actions and Interactions
• Domain Adaptation by Maximizing Population Correlation with Neural Architecture Search
• Learning to see across Domains and Modalities
• Offer Personalization using Temporal Convolution Network and Optimization
• PrivNet: Safeguarding Private Attributes in Transfer Learning for Recommendation
• Matching the Clinical Reality: Accurate OCT-Based Diagnosis From Few Labels
• Does Optimal Source Task Performance Imply Optimal Pre-training for a Target Task?
• Learning Super-resolution 3D Segmentation of Plant Root MRI Images from Few Examples
• Canoe : A System for Collaborative Learning for Neural Nets
• Learning Neural Models for Continuous-Time Sequences
• EE-AE: An Exclusivity Enhanced Unsupervised Feature Learning Approach
• Towards All-around Knowledge Transferring: Learning From Task-irrelevant Labels
• NI-UDA: Graph Adversarial Domain Adaptation from Non-shared-and-Imbalanced Big Data to Small Imbalanced Applications
• Knowledge Distillation from BERT Transformer to Speech Transformer for Intent Classification
• Sex-Prediction from Periocular Images across Multiple Sensors and Spectra
• ISS-MULT: Intelligent Sample Selection for Multi-Task Learning in Question Answering
• A Survey on Green Deep Learning
• Incorporating Human Domain Knowledge in 3D LiDAR-based Semantic Segmentation
• Integration of Text-maps in Convolutional Neural Networks for Region Detection among Different Textual Categories
• Reinforced Temporal Attention and Split-Rate Transfer for Depth-Based Person Re-Identification
• Domain Adaptor Networks for Hyperspectral Image Recognition
• Network Implosion: Effective Model Compression for ResNets via Static Layer Pruning and Retraining
• Self-Tuning for Data-Efficient Deep Learning
• Transfer Reinforcement Learning across Homotopy Classes
• Effects of Layer Freezing on Transferring a Speech Recognition System to Under-resourced Languages
• Combine PPO with NES to Improve Exploration
• Application of Deep Convolutional Neural Networks for automated and rapid identification and characterization of thin cracks in SHCCs
• Learning to Generalize One Sample at a Time with Self-Supervision
• IIITM Face: A Database for Facial Attribute Detection in Constrained and Simulated Unconstrained Environments
• DSRGAN: Explicitly Learning Disentangled Representation of Underlying Structure and Rendering for Image Generation without Tuple Supervision
• High Accurate Time-of-Arrival Estimation with Fine-Grained Feature Generation for Internet-of-Things Applications
• A Neural-Symbolic Framework for Mental Simulation
• Transferable End-to-End Aspect-based Sentiment Analysis with Selective Adversarial Learning
• Universality of Gradient Descent Neural Network Training
• Across-domains transferability of Deep-RED in de-noising and compressive sensing recovery of seismic data
• A Simple Domain Shifting Networkfor Generating Low Quality Images
• A Deep Learning Approach for Robust Corridor Following