Distilling the Knowledge in a Neural Network

Abstract

A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions. Unfortunately, making predictions using a whole ensemble of models is cumbersome and may be too computationally expensive to allow deployment to a large number of users, especially if the individual models are large neural nets. Caruana and his collaborators have shown that it is possible to compress the knowledge in an ensemble into a single model which is much easier to deploy and we develop this approach further using a different compression technique. We achieve some surprising results on MNIST and we show that we can significantly improve the acoustic model of a heavily used commercial system by distilling the knowledge in an ensemble of models into a single model. We also introduce a new type of ensemble composed of one or more full models and many specialist models which learn to distinguish fine-grained classes that the full models confuse. Unlike a mixture of experts, these specialist models can be trained rapidly and in parallel.

NIPS 2014 Deep Learning Workshop

References in corpus (1)

• Improving neural networks by preventing co-adaptation of feature detectors

Cited by in corpus (3127)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
• A Brief Survey of Deep Reinforcement Learning
• A Survey on Visual Transformer
• Knowledge Distillation: A Survey
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• Transformers in Vision: A Survey
• Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
• Language Models are Few-Shot Learners
• ICLabel: An automated electroencephalographic independent component classifier, dataset, and website
• On the Opportunities and Risks of Foundation Models
• Deep Visual Domain Adaptation: A Survey
• Ensemble deep learning: A review
• ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
• On Calibration of Modern Neural Networks
• AutoML: A Survey of the State-of-the-Art
• A continual learning survey: Defying forgetting in classification tasks
• Unsupervised Data Augmentation for Consistency Training
• Pre-trained Models for Natural Language Processing: A Survey
• Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
• Temporal Ensembling for Semi-Supervised Learning
• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Semantic Foggy Scene Understanding with Synthetic Data
• The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
• Weight Uncertainty in Neural Networks
• Towards Personalized Federated Learning
• Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
• DINOv2: Learning Robust Visual Features without Supervision
• SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
• A Survey on Deep Semi-supervised Learning
• ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
• Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users
• Linformer: Self-Attention with Linear Complexity
• When Does Label Smoothing Help?
• A Survey of Model Compression and Acceleration for Deep Neural Networks
• Medical Image Segmentation Using Deep Learning: A Survey
• Align before Fuse: Vision and Language Representation Learning with Momentum Distillation
• How To Backdoor Federated Learning
• Multi-Task Learning for Dense Prediction Tasks: A Survey
• ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
• Embedding Watermarks into Deep Neural Networks
• A Survey and Critique of Multiagent Deep Reinforcement Learning
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
• Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
• CyCADA: Cycle-Consistent Adversarial Domain Adaptation
• FedKD: Communication Efficient Federated Learning via Knowledge Distillation
• Human Action Recognition from Various Data Modalities: A Review
• Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challenges
• Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better
• Compressing Neural Networks with the Hashing Trick
• Object Detection in 20 Years: A Survey
• Three scenarios for continual learning
• Binary Neural Networks: A Survey
• Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes
• Ensemble Distillation for Robust Model Fusion in Federated Learning
• FedMD: Heterogenous Federated Learning via Model Distillation
• Data Augmentation using Random Image Cropping and Patching for Deep CNNs
• Group Sparse Regularization for Deep Neural Networks
• Joint Classification and Prediction CNN Framework for Automatic Sleep Stage Classification
• Big Self-Supervised Models are Strong Semi-Supervised Learners
• A Review on Deep Learning in UAV Remote Sensing
• Adversarial Attacks and Defences: A Survey
• Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels
• Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data
• Born Again Neural Networks
• The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale
• Well-Read Students Learn Better: On the Importance of Pre-training Compact Models
• Interpretability of machine learning based prediction models in healthcare
• A Review of Generalized Zero-Shot Learning Methods
• Avoiding Overfitting: A Survey on Regularization Methods for Convolutional Neural Networks
• Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification
• Adversarial Examples Are Not Bugs, They Are Features
• Visual Transformers: Token-based Image Representation and Processing for Computer Vision
• Knowledge Distillation by On-the-Fly Native Ensemble
• Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
• Xception: Deep Learning with Depthwise Separable Convolutions
• Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Data
• Predictive Uncertainty Estimation via Prior Networks
• Image Classification with Deep Learning in the Presence of Noisy Labels: A Survey
• Like What You Like: Knowledge Distill via Neuron Selectivity Transfer
• Bayesian Compression for Deep Learning
• Distilling Task-Specific Knowledge from BERT into Simple Neural Networks
• Billion-scale semi-supervised learning for image classification
• On Tiny Episodic Memories in Continual Learning
• Annotation-efficient deep learning for automatic medical image segmentation
• A Review on Deep Learning Techniques for Video Prediction
• DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification
• R-Drop: Regularized Dropout for Neural Networks
• Intriguing Properties of Vision Transformers
• Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
• Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges
• Multi-source Transfer Learning with Convolutional Neural Networks for Lung Pattern Analysis
• TransNets: Learning to Transform for Recommendation
• Learned Step Size Quantization
• Class-Incremental Learning: A Survey
• Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models
• One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques
• DeepNano: Deep Recurrent Neural Networks for Base Calling in MinION Nanopore Reads
• Distilling a Neural Network Into a Soft Decision Tree
• Sound Event Detection: A Tutorial
• Towards a Human-like Open-Domain Chatbot
• BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer
• What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use
• DeeperGCN: All You Need to Train Deeper GCNs
• Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey
• A Primer on Motion Capture with Deep Learning: Principles, Pitfalls and Perspectives
• A Benchmark Study of Machine Learning Models for Online Fake News Detection
• Self-training with Noisy Student improves ImageNet classification
• The Principles of Deep Learning Theory
• Leveraging Recent Advances in Deep Learning for Audio-Visual Emotion Recognition
• Slimmable Neural Networks
• Deep Learning for Generic Object Detection: A Survey
• SoundNet: Learning Sound Representations from Unlabeled Video
• Generating Adversarial Examples with Adversarial Networks
• Adaptive Aggregation Networks for Class-Incremental Learning
• Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI
• Adaptive Multi-Teacher Multi-level Knowledge Distillation
• Adversarial Examples: Attacks and Defenses for Deep Learning
• Low-resolution Face Recognition in the Wild via Selective Knowledge Distillation
• Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead
• Data-Free Knowledge Distillation for Deep Neural Networks
• DocBERT: BERT for Document Classification
• A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks
• EVA-02: A Visual Representation for Neon Genesis
• Do Convolutional Neural Networks Learn Class Hierarchy?
• Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis
• A Survey on Neural Architecture Search
• Bad Actor, Good Advisor: Exploring the Role of Large Language Models in Fake News Detection
• Improving Multi-hop Knowledge Base Question Answering by Learning Intermediate Supervision Signals
• All you need is a good init
• A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?
• A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions
• Designing Interpretable ML System to Enhance Trust in Healthcare: A Systematic Review to Proposed Responsible Clinician-AI-Collaboration Framework
• Harnessing Deep Neural Networks with Logic Rules
• Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning
• Efficient Medical Image Segmentation Based on Knowledge Distillation
• Bayesian Dark Knowledge
• A Survey of Domain Adaptation for Neural Machine Translation
• TADAM: Task dependent adaptive metric for improved few-shot learning
• Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge
• SPICE: Semantic Pseudo-labeling for Image Clustering
• Temporal 3D ConvNets: New Architecture and Transfer Learning for Video Classification
• UNITER: UNiversal Image-TExt Representation Learning
• A Survey on Neural Speech Synthesis
• Regularization for Deep Learning: A Taxonomy
• Incorporating Domain Knowledge into Deep Neural Networks
• A Review of Some Techniques for Inclusion of Domain-Knowledge into Deep Neural Networks
• RMDL: Recalibrated multi-instance deep learning for whole slide gastric image classification
• Distral: Robust Multitask Reinforcement Learning
• Deformable ConvNets v2: More Deformable, Better Results
• Generative replay with feedback connections as a general strategy for continual learning
• Semantic Image Segmentation via Deep Parsing Network
• Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling
• WORD: A large scale dataset, benchmark and clinical applicable study for abdominal organ segmentation from CT image
• Dynamic Convolutions: Exploiting Spatial Sparsity for Faster Inference
• Tiny Machine Learning: Progress and Futures
• Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination
• Class-specific Differential Detection in Diffractive Optical Neural Networks Improves Inference Accuracy
• Bringing AI To Edge: From Deep Learning's Perspective
• A Transformer-Based Model With Self-Distillation for Multimodal Emotion Recognition in Conversations
• Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
• The NLP Cookbook: Modern Recipes for Transformer based Deep Learning Architectures
• Adversarial Machine Learning in Image Classification: A Survey Towards the Defender's Perspective
• Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding
• Universal Domain Adaptation through Self Supervision
• Verifiable Reinforcement Learning via Policy Extraction
• Expanding the Reach of Federated Learning by Reducing Client Resource Requirements
• Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach
• Explainable Deep Reinforcement Learning: State of the Art and Challenges
• Unifying distillation and privileged information
• Knowledge Guided Disambiguation for Large-Scale Scene Classification with Multi-Resolution CNNs
• ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias
• Net2Net: Accelerating Learning via Knowledge Transfer
• Large scale distributed neural network training through online distillation
• Transfer Learning in Deep Reinforcement Learning: A Survey
• Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning
• Label Refinery: Improving ImageNet Classification through Label Progression
• Path-Level Network Transformation for Efficient Architecture Search
• Incremental Learning for Semantic Segmentation of Large-Scale Remote Sensing Data
• Distilled Siamese Networks for Visual Tracking
• Keyword Transformer: A Self-Attention Model for Keyword Spotting
• MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers
• Towards Fairness-Aware Federated Learning
• An Entropy-based Pruning Method for CNN Compression
• Automatic Sleep Staging of EEG Signals: Recent Development, Challenges, and Future Directions
• All Tokens Matter: Token Labeling for Training Better Vision Transformers
• DSD: Dense-Sparse-Dense Training for Deep Neural Networks
• Bag of Tricks for Image Classification with Convolutional Neural Networks
• Stabilizing the Lottery Ticket Hypothesis
• One-Shot Federated Learning
• Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks
• Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications
• Hardware Approximate Techniques for Deep Neural Network Accelerators: A Survey
• A Deep Hierarchical Approach to Lifelong Learning in Minecraft
• Network Pruning via Transformable Architecture Search
• A Practical Survey on Faster and Lighter Transformers
• TinyBERT: Distilling BERT for Natural Language Understanding
• Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog
• A survey on efficient vision transformers: algorithms, techniques, and performance benchmarking
• Compression-aware Training of Deep Networks
• Thinking Fast and Slow: Efficient Text-to-Visual Retrieval with Transformers
• Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples
• Carbon Emissions and Large Neural Network Training
• SelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Data
• 6G White Paper on Edge Intelligence
• Edge Intelligence: Architectures, Challenges, and Applications
• Continual Learning for Recurrent Neural Networks: an Empirical Evaluation
• Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation
• Interventional Few-Shot Learning
• Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANs
• GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose
• Random Feature Attention
• Decision-Focused Learning: Foundations, State of the Art, Benchmark and Future Opportunities
• Learning Intrinsic Sparse Structures within Long Short-Term Memory
• Variational Federated Multi-Task Learning
• End-to-End Object Detection with Adaptive Clustering Transformer
• Federated Continual Learning via Knowledge Fusion: A Survey
• Reinforcement Learning in Healthcare: A Survey
• Universal representations:The missing link between faces, text, planktons, and cat breeds
• N2N Learning: Network to Network Compression via Policy Gradient Reinforcement Learning
• Knowledge Distillation in Deep Learning and its Applications
• GhostNet: More Features from Cheap Operations
• Training with Quantization Noise for Extreme Model Compression
• Supervised Contrastive Learning
• Recent Advances in Deep Learning: An Overview
• FasterSeg: Searching for Faster Real-time Semantic Segmentation
• Federated Learning for Computationally-Constrained Heterogeneous Devices: A Survey
• Distilling Knowledge from Deep Networks with Applications to Healthcare Domain
• Face Behavior a la carte: Expressions, Affect and Action Units in a Single Network
• Compressing Recurrent Neural Network with Tensor Train
• Recurrent Neural Network Training with Dark Knowledge Transfer
• Light-Weight RefineNet for Real-Time Semantic Segmentation
• CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation
• Sequence-Level Knowledge Distillation
• Lifelong Generative Modeling
• Scene Text Detection and Recognition: The Deep Learning Era
• Unsupervised Domain Adaptation with Adversarial Residual Transform Networks
• Conditional Teacher-Student Learning
• Projective Inference in High-dimensional Problems: Prediction and Feature Selection
• Pretrained Transformers for Text Ranking: BERT and Beyond
• Unsupervised Scene Adaptation with Memory Regularization in vivo
• Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions
• Are All Layers Created Equal?
• Gradient Surgery for Multi-Task Learning
• Dataset Condensation with Gradient Matching
• FedCP: Separating Feature Information for Personalized Federated Learning via Conditional Policy
• Incremental Object Detection via Meta-Learning
• Do Deep Convolutional Nets Really Need to be Deep and Convolutional?
• HARDC : A novel ECG-based heartbeat classification method to detect arrhythmia using hierarchical attention based dual structured RNN with dilated CNN
• Data-Free Adversarial Distillation
• A Teacher-Student Framework for Zero-Resource Neural Machine Translation
• From Facial Parts Responses to Face Detection: A Deep Learning Approach
• Outlier Exposure with Confidence Control for Out-of-Distribution Detection
• Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Study
• Once-for-All: Train One Network and Specialize it for Efficient Deployment
• Structured Transforms for Small-Footprint Deep Learning
• Continual Learning in Generative Adversarial Nets
• Focus: Querying Large Video Datasets with Low Latency and Low Cost
• Spot-adaptive Knowledge Distillation
• Self-Distillation Amplifies Regularization in Hilbert Space
• Learning Lightweight Lane Detection CNNs by Self Attention Distillation
• Learning to Learn: Meta-Critic Networks for Sample Efficient Learning
• Practical Black-Box Attacks against Machine Learning
• Review of data analysis in vision inspection of power lines with an in-depth discussion of deep learning technology
• CondenseNet: An Efficient DenseNet using Learned Group Convolutions
• MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
• DELTA: DEep Learning Transfer using Feature Map with Attention for Convolutional Networks
• Pseudo-Rehearsal: Achieving Deep Reinforcement Learning without Catastrophic Forgetting
• Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift
• Soft-Label Dataset Distillation and Text Dataset Distillation
• Incremental Classifier Learning with Generative Adversarial Networks
• On the Effect of Dropping Layers of Pre-trained Transformer Models
• Learning Semantically Enhanced Feature for Fine-Grained Image Classification
• Iterative Machine Teaching
• BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning
• Adapting Auxiliary Losses Using Gradient Similarity
• Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer
• No One Left Behind: Inclusive Federated Learning over Heterogeneous Devices
• Practical Blind Membership Inference Attack via Differential Comparisons
• Dark Experience for General Continual Learning: a Strong, Simple Baseline
• A Systematic Review on Model Watermarking for Neural Networks
• Reward Augmented Maximum Likelihood for Neural Structured Prediction
• What Do Compressed Deep Neural Networks Forget?
• Double Similarity Distillation for Semantic Image Segmentation
• Stealing Neural Networks via Timing Side Channels
• Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?
• Understanding and Improving Knowledge Distillation
• Learning with privileged information via adversarial discriminative modality distillation
• Paraphrasing Complex Network: Network Compression via Factor Transfer
• Dynamic Sampling Networks for Efficient Action Recognition in Videos
• Vision Transformers, a new approach for high-resolution and large-scale mapping of canopy heights
• Object Detection with Deep Learning: A Review
• Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation
• CheXtransfer: Performance and Parameter Efficiency of ImageNet Models for Chest X-Ray Interpretation
• Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model
• Deep Graph-level Anomaly Detection by Glocal Knowledge Distillation
• Secure and Trustworthy Artificial Intelligence-Extended Reality (AI-XR) for Metaverses
• Big-Little Net: An Efficient Multi-Scale Feature Representation for Visual and Speech Recognition
• Multi-task learning for natural language processing in the 2020s: where are we going?
• Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation
• Explainable Neural Networks based on Additive Index Models
• Extending Defensive Distillation
• Environmental Sound Classification on the Edge: A Pipeline for Deep Acoustic Networks on Extremely Resource-Constrained Devices
• Artificial Neural Networks for Photonic Applications: From Algorithms to Implementation
• FIgLib & SmokeyNet: Dataset and Deep Learning Model for Real-Time Wildland Fire Smoke Detection
• MobiSR: Efficient On-Device Super-Resolution through Heterogeneous Mobile Processors
• Labelling unlabelled videos from scratch with multi-modal self-supervision
• Margin Preserving Self-paced Contrastive Learning Towards Domain Adaptation for Medical Image Segmentation
• Unsupervised Embedding Learning via Invariant and Spreading Instance Feature
• APE-GAN: Adversarial Perturbation Elimination with GAN
• Feature-map-level Online Adversarial Knowledge Distillation
• Adaptation Algorithms for Neural Network-Based Speech Recognition: An Overview
• Deep Long-Tailed Learning: A Survey
• Cross-Resolution Learning for Face Recognition
• Supervised Compression for Resource-Constrained Edge Computing Systems
• Privacy-Preserving Machine Learning: Methods, Challenges and Directions
• Network Decoupling: From Regular to Depthwise Separable Convolutions
• Applications and Techniques for Fast Machine Learning in Science
• A Low Effort Approach to Structured CNN Design Using PCA
• Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems
• UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation
• K-Core based Temporal Graph Convolutional Network for Dynamic Graphs
• CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition
• Deep Learning for Sequential Recommendation: Algorithms, Influential Factors, and Evaluations
• GraphSAIL: Graph Structure Aware Incremental Learning for Recommender Systems
• Hierarchical Self-supervised Augmented Knowledge Distillation
• Few-Shot Text Generation with Pattern-Exploiting Training
• SEED: Self-supervised Distillation For Visual Representation
• Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch
• FedCG: Leverage Conditional GAN for Protecting Privacy and Maintaining Competitive Performance in Federated Learning
• RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering
• Distilled One-Shot Federated Learning
• On the Expressive Power of Deep Neural Networks
• LDMVFI: Video Frame Interpolation with Latent Diffusion Models
• Eye Tracking for Everyone
• Self-supervised Knowledge Distillation for Few-shot Learning
• Federated Mutual Learning
• Comprehensive Attention Self-Distillation for Weakly-Supervised Object Detection
• The unreasonable effectiveness of the forget gate
• DyNet: Dynamic Convolution for Accelerating Convolutional Neural Networks
• Training Binary Neural Networks with Real-to-Binary Convolutions
• Salvaging Federated Learning by Local Adaptation
• A Closer Look at Deep Learning Heuristics: Learning rate restarts, Warmup and Distillation
• Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimization
• Knowledge Distillation in Generations: More Tolerant Teachers Educate Better Students
• NBDT: Neural-Backed Decision Trees
• Federated Learning for Healthcare Informatics
• Urban Land Cover Classification with Missing Data Modalities Using Deep Convolutional Neural Networks
• Rethinking Spatial Dimensions of Vision Transformers
• Efficient Multi-objective Neural Architecture Search via Lamarckian Evolution
• Should Graph Convolution Trust Neighbors? A Simple Causal Inference Method
• Continual Learning in Sensor-based Human Activity Recognition: an Empirical Benchmark Analysis
• Deep learning as a tool for neural data analysis: speech classification and cross-frequency coupling in human sensorimotor cortex
• Promises and pitfalls of deep neural networks in neuroimaging-based psychiatric research
• KERMIT: Generative Insertion-Based Modeling for Sequences
• Learning to Disentangle Scenes for Person Re-identification
• On Representation Knowledge Distillation for Graph Neural Networks
• And the Bit Goes Down: Revisiting the Quantization of Neural Networks
• Hurricane Forecasting: A Novel Multimodal Machine Learning Framework
• Contrastive Representation Distillation
• SpaceNet: Make Free Space For Continual Learning
• Alexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems
• Domain Robustness in Neural Machine Translation
• FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout
• Improved Recurrent Neural Networks for Session-based Recommendations
• Prediction Poisoning: Towards Defenses Against DNN Model Stealing Attacks
• EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples
• Neural Compression and Filtering for Edge-assisted Real-time Object Detection in Challenged Networks
• Lifelong Teacher-Student Network Learning
• Enhancing Video-Language Representations with Structural Spatio-Temporal Alignment
• Consistency-Regularized Region-Growing Network for Semantic Segmentation of Urban Scenes with Point-Level Annotations
• ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech
• NISP: Pruning Networks using Neuron Importance Score Propagation
• A Survey on Deep Neural Network Compression: Challenges, Overview, and Solutions
• ResKD: Residual-Guided Knowledge Distillation
• Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed
• Ensemble Distillation for Neural Machine Translation
• Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventions
• Compressing GANs using Knowledge Distillation
• Fast Structured Decoding for Sequence Models
• Sobolev Training for Neural Networks
• Great Power, Great Responsibility: Recommendations for Reducing Energy for Training Language Models
• Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning
• Synthetic Data Generation Using Large Language Models: Advances in Text and Code
• Self-Supervised Monocular Depth Estimation with Self-Reference Distillation and Disparity Offset Refinement
• Security and Privacy Issues in Deep Learning
• MRI-based Alzheimer's disease prediction via distilling the knowledge in multi-modal data
• Multi-modal Deep Analysis for Multimedia
• EDropout: Energy-Based Dropout and Pruning of Deep Neural Networks
• Towards Utilizing Unlabeled Data in Federated Learning: A Survey and Prospective
• Pre-trained Summarization Distillation
• Distilling Dense Representations for Ranking using Tightly-Coupled Teachers
• Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers
• DeCoAR 2.0: Deep Contextualized Acoustic Representations with Vector Quantization
• Compact recurrent neural networks for acoustic event detection on low-energy low-complexity platforms
• Learning from Noisy Labels with Distillation
• Graph-less Collaborative Filtering
• Improving Fast Segmentation With Teacher-student Learning
• On Transferability of Prompt Tuning for Natural Language Processing
• FedICT: Federated Multi-task Distillation for Multi-access Edge Computing
• FastBERT: a Self-distilling BERT with Adaptive Inference Time
• Visual Relationship Detection with Internal and External Linguistic Knowledge Distillation
• BottleFit: Learning Compressed Representations in Deep Neural Networks for Effective and Efficient Split Computing
• Hybrid Models for Learning to Branch
• Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons
• FermiNets: Learning generative machines to generate efficient neural networks via generative synthesis
• Small Sample Learning in Big Data Era
• Lets keep it simple, Using simple architectures to outperform deeper and more complex architectures
• Transferring Knowledge from a RNN to a DNN
• Testing and verification of neural-network-based safety-critical control software: A systematic literature review
• Overcoming Long-term Catastrophic Forgetting through Adversarial Neural Pruning and Synaptic Consolidation
• A Survey on Transfer Learning in Natural Language Processing
• SNIPER: Efficient Multi-Scale Training
• Uses and Abuses of the Cross-Entropy Loss: Case Studies in Modern Deep Learning
• Compacting Deep Neural Networks for Internet of Things: Methods and Applications
• Theory and Experiments on Vector Quantized Autoencoders
• High Accuracy and High Fidelity Extraction of Neural Networks
• A Survey of Safety and Trustworthiness of Deep Neural Networks: Verification, Testing, Adversarial Attack and Defence, and Interpretability
• OSLNet: Deep Small-Sample Classification with an Orthogonal Softmax Layer
• Efficient Convolutional Neural Networks for Depth-Based Multi-Person Pose Estimation
• Lightweight Pixel Difference Networks for Efficient Visual Representation Learning
• Salient Objects in Clutter
• Uncertainty Aware Semi-Supervised Learning on Graph Data
• Vision Transformer Pruning
• Vision-Language Models for Edge Networks: A Comprehensive Survey
• Cross-dimensional transfer learning in medical image segmentation with deep learning
• Dataset Condensation with Differentiable Siamese Augmentation
• Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking
• Learning From Noisy Large-Scale Datasets With Minimal Supervision
• Real-time Action Recognition with Enhanced Motion Vector CNNs
• Long-tail learning via logit adjustment
• Self-Distillation as Instance-Specific Label Smoothing
• Neural Network Distiller: A Python Package For DNN Compression Research
• Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT
• RobustART: Benchmarking Robustness on Architecture Design and Training Techniques
• Learning from others' mistakes: Avoiding dataset biases without modeling them
• Introduction to Camera Pose Estimation with Deep Learning
• Compositional Explanations of Neurons
• Adaptive Boosting for Domain Adaptation: Towards Robust Predictions in Scene Segmentation
• Adversarial Examples in Modern Machine Learning: A Review
• Object DGCNN: 3D Object Detection using Dynamic Graphs
• Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN)
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
• MEAL V2: Boosting Vanilla ResNet-50 to 80%+ Top-1 Accuracy on ImageNet without Tricks
• ModelDiff: Testing-Based DNN Similarity Comparison for Model Reuse Detection
• On-Device Machine Learning: An Algorithms and Learning Theory Perspective
• Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection
• HRel: Filter Pruning based on High Relevance between Activation Maps and Class Labels
• MobileFaceNets: Efficient CNNs for Accurate Real-Time Face Verification on Mobile Devices
• EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentation
• Efficient Continual Learning in Neural Networks with Embedding Regularization
• Relational Knowledge Distillation
• Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data
• BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction
• Learn the new, keep the old: Extending pretrained models with new anatomy and images
• Cognitively-Inspired Model for Incremental Learning Using a Few Examples
• ST-MFNet: A Spatio-Temporal Multi-Flow Network for Frame Interpolation
• Diversity Networks: Neural Network Compression Using Determinantal Point Processes
• Efficient Processing of Deep Neural Networks: A Tutorial and Survey
• Deep Mutual Learning
• Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion
• Self-Augmentation: Generalizing Deep Networks to Unseen Classes for Few-Shot Learning
• Learning Implicit Credit Assignment for Cooperative Multi-Agent Reinforcement Learning
• Implementation of a Modified U-Net for Medical Image Segmentation on Edge Devices
• QKD: Quantization-aware Knowledge Distillation
• Self-Supervised Pre-Training for Transformer-Based Person Re-Identification
• Model Rubik's Cube: Twisting Resolution, Depth and Width for TinyNets
• On Interpretability of Artificial Neural Networks: A Survey
• NoScope: Optimizing Neural Network Queries over Video at Scale
• Synergic Adversarial Label Learning for Grading Retinal Diseases via Knowledge Distillation and Multi-task Learning
• Self-training Improves Pre-training for Natural Language Understanding
• DeLighT: Deep and Light-weight Transformer
• Hybrid Tensor Decomposition in Neural Network Compression
• 2PFPCE: Two-Phase Filter Pruning Based on Conditional Entropy
• ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models
• Learning Student-Friendly Teacher Networks for Knowledge Distillation
• BERT Loses Patience: Fast and Robust Inference with Early Exit
• Fine-Grained Fashion Similarity Prediction by Attribute-Specific Embedding Learning
• Rethinking Soft Labels for Knowledge Distillation: A Bias-Variance Tradeoff Perspective
• Large image datasets: A pyrrhic win for computer vision?
• Understanding and Improving Lexical Choice in Non-Autoregressive Translation
• A Survey on Symbolic Knowledge Distillation of Large Language Models
• Adversarial Examples - A Complete Characterisation of the Phenomenon
• Learning Approximate Inference Networks for Structured Prediction
• KD3A: Unsupervised Multi-Source Decentralized Domain Adaptation via Knowledge Distillation
• Distilling Object Detectors with Task Adaptive Regularization
• Debiasing Methods for Fairer Neural Models in Vision and Language Research: A Survey
• Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation
• Towards Unconstrained Palmprint Recognition on Consumer Devices: a Literature Review
• Learning Target Domain Specific Classifier for Partial Domain Adaptation
• On the Compression of Recurrent Neural Networks with an Application to LVCSR acoustic modeling for Embedded Speech Recognition
• Data Diversification: A Simple Strategy For Neural Machine Translation
• Data Distillation: Towards Omni-Supervised Learning
• Only Train Once: A One-Shot Neural Network Training And Pruning Framework
• Federated Learning Meets Natural Language Processing: A Survey
• A deep architecture for unified aesthetic prediction
• A Survey on Continual Semantic Segmentation: Theory, Challenge, Method and Application
• DER: Dynamically Expandable Representation for Class Incremental Learning
• Mobile Video Object Detection with Temporally-Aware Feature Maps
• M2KD: Multi-model and Multi-level Knowledge Distillation for Incremental Learning
• Self-supervised transfer learning of physiological representations from free-living wearable data
• A Strategy for an Uncompromising Incremental Learner
• Chess AI: Competing Paradigms for Machine Intelligence
• Distilling Knowledge by Mimicking Features
• Adversarial Patch Attacks on Monocular Depth Estimation Networks
• PoPS: Policy Pruning and Shrinking for Deep Reinforcement Learning
• TRILLsson: Distilled Universal Paralinguistic Speech Representations
• ReSSL: Relational Self-Supervised Learning with Weak Augmentation
• Knowledge Adaptation: Teaching to Adapt
• LTE4G: Long-Tail Experts for Graph Neural Networks
• ExpandNets: Linear Over-parameterization to Train Compact Convolutional Networks
• Direct Speech-to-image Translation
• Label Noise Types and Their Effects on Deep Learning
• Improved training of binary networks for human pose estimation and image recognition
• Detecting Outliers with Foreign Patch Interpolation
• A Post-processing Method for Detecting Unknown Intent of Dialogue System via Pre-trained Deep Neural Network Classifier
• Bridging the Modality Gap for Speech-to-Text Translation
• Knowledge Transfer Between Artificial Intelligence Systems
• Incorporating BERT into Parallel Sequence Decoding with Adapters
• Enhancing Diversity in Teacher-Student Networks via Asymmetric branches for Unsupervised Person Re-identification
• Kickstarting Deep Reinforcement Learning
• Dynamic Neural Networks: A Survey
• BlockDrop: Dynamic Inference Paths in Residual Networks
• DeepIoT: Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Framework
• End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things
• Distilling Policy Distillation
• Orthogonal Gradient Descent for Continual Learning
• BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Models
• Cream of the Crop: Distilling Prioritized Paths For One-Shot Neural Architecture Search
• Deep Learning with Logical Constraints
• AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processing
• Knowledge Distillation approach towards Melanoma Detection
• Evaluating Explanation Without Ground Truth in Interpretable Machine Learning
• Looking for the Devil in the Details: Learning Trilinear Attention Sampling Network for Fine-grained Image Recognition
• FaceNet2ExpNet: Regularizing a Deep Face Recognition Net for Expression Recognition
• Semantics-Native Communication with Contextual Reasoning
• Unified Probabilistic Deep Continual Learning through Generative Replay and Open Set Recognition
• Graph-based Knowledge Distillation by Multi-head Attention Network
• Learning to Learn from Weak Supervision by Full Supervision
• Heated-Up Softmax Embedding
• End-to-End Learning of Semantic Grasping
• Complexity-Driven CNN Compression for Resource-constrained Edge AI
• Can You Trust This Prediction? Auditing Pointwise Reliability After Learning
• Unsupervised Learning on Neural Network Outputs: with Application in Zero-shot Learning
• Dimensionality Reduction using Similarity-induced Embeddings
• Adversarial Generation of Training Examples: Applications to Moving Vehicle License Plate Recognition
• Trading-off Accuracy and Energy of Deep Inference on Embedded Systems: A Co-Design Approach
• Transfer Learning for Future Wireless Networks: A Comprehensive Survey
• Continual Learning for Generative Retrieval over Dynamic Corpora
• More is Less: A More Complicated Network with Less Inference Complexity
• Clustering-friendly Representation Learning via Instance Discrimination and Feature Decorrelation
• Local-Global Knowledge Distillation in Heterogeneous Federated Learning with Non-IID Data
• One-shot Federated Learning without Server-side Training
• RescoreBERT: Discriminative Speech Recognition Rescoring with BERT
• Deep Person Generation: A Survey from the Perspective of Face, Pose and Cloth Synthesis
• Distilling Object Detectors with Feature Richness
• Knowledge Distillation via Route Constrained Optimization
• End-to-End Speech Translation with Knowledge Distillation
• Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation
• Low-Memory Neural Network Training: A Technical Report
• Contrastive Learning with Stronger Augmentations
• Compounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network
• XPipe: Efficient Pipeline Model Parallelism for Multi-GPU DNN Training
• MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features
• HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision
• Pixel-wise Attentional Gating for Parsimonious Pixel Labeling
• Continual Learning for Fake Audio Detection
• Generation & Evaluation of Adversarial Examples for Malware Obfuscation
• Variational Information Distillation for Knowledge Transfer
• Learning Contrastive Self-Distillation for Ultra-Fine-Grained Visual Categorization Targeting Limited Samples
• Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data
• Hydra: Preserving Ensemble Diversity for Model Distillation
• Dynamically Expandable Graph Convolution for Streaming Recommendation
• DecomposeMe: Simplifying ConvNets for End-to-End Learning
• Training Binary Neural Networks through Learning with Noisy Supervision
• BERT-of-Theseus: Compressing BERT by Progressive Module Replacing
• Multi-Task Incremental Learning for Object Detection
• Toward Fast and Accurate Neural Chinese Word Segmentation with Multi-Criteria Learning
• Multimodal Fish Feeding Intensity Assessment in Aquaculture
• Predicting Deeper into the Future of Semantic Segmentation
• Online Hyperparameter Optimization for Class-Incremental Learning
• Pruning vs XNOR-Net: A Comprehensive Study of Deep Learning for Audio Classification on Edge-devices
• A distillation-based approach integrating continual learning and federated learning for pervasive services
• Deep Reinforcement Learning with Successor Features for Navigation across Similar Environments
• Radioactive data: tracing through training
• Cross-domain Self-supervised Learning for Domain Adaptation with Few Source Labels
• PRADA: Protecting against DNN Model Stealing Attacks
• Bridging the Gap Between Patient-specific and Patient-independent Seizure Prediction via Knowledge Distillation
• Towards Better Accuracy-efficiency Trade-offs: Divide and Co-training
• Considerations When Learning Additive Explanations for Black-Box Models
• Honest-but-Curious Nets: Sensitive Attributes of Private Inputs Can Be Secretly Coded into the Classifiers' Outputs
• Movement Pruning: Adaptive Sparsity by Fine-Tuning
• Confidence Regularized Self-Training
• End-to-End Adversarial Text-to-Speech
• End-to-End Incremental Learning
• TinyML for Ubiquitous Edge AI
• Learning Spatiotemporal Features via Video and Text Pair Discrimination
• Zero-shot Knowledge Transfer via Adversarial Belief Matching
• KGNN: Harnessing Kernel-based Networks for Semi-supervised Graph Classification
• Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directions
• Satellite Monitoring of Terrestrial Plastic Waste
• Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Text
• HyPHEN: A Hybrid Packing Method and Optimizations for Homomorphic Encryption-Based Neural Networks
• Towards Optimal Structured CNN Pruning via Generative Adversarial Learning
• FedHe: Heterogeneous Models and Communication-Efficient Federated Learning
• Understanding Knowledge Distillation in Non-autoregressive Machine Translation
• SCSP: Spectral Clustering Filter Pruning with Soft Self-adaption Manners
• Multi-Task Zipping via Layer-wise Neuron Sharing
• SUOD: Accelerating Large-Scale Unsupervised Heterogeneous Outlier Detection
• Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning
• Feature Fusion for Online Mutual Knowledge Distillation
• Pseudo-LiDAR Based Road Detection
• The Evolution of Reinforcement Learning in Quantitative Finance: A Survey
• Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translation
• Maintaining Discrimination and Fairness in Class Incremental Learning
• Unsupervised Cross-Modal Distillation for Thermal Infrared Tracking
• ChipNet: Budget-Aware Pruning with Heaviside Continuous Approximations
• Contrastive Model Inversion for Data-Free Knowledge Distillation
• Composition of Deep and Spiking Neural Networks for Very Low Bit Rate Speech Coding
• DSConv: Efficient Convolution Operator
• Towards Explainable Graph Representations in Digital Pathology
• Zero Time Waste: Recycling Predictions in Early Exit Neural Networks
• AANet: Adaptive Aggregation Network for Efficient Stereo Matching
• Dream Distillation: A Data-Independent Model Compression Framework
• Long-tailed Recognition by Routing Diverse Distribution-Aware Experts
• Scene Graph Generation with External Knowledge and Image Reconstruction
• Spatio-Temporal Graph for Video Captioning with Knowledge Distillation
• FreeKD: Free-direction Knowledge Distillation for Graph Neural Networks
• FedACK: Federated Adversarial Contrastive Knowledge Distillation for Cross-Lingual and Cross-Model Social Bot Detection
• Visualizing and Understanding Curriculum Learning for Long Short-Term Memory Networks
• Stochastic Multiple Choice Learning for Training Diverse Deep Ensembles
• Inherit with Distillation and Evolve with Contrast: Exploring Class Incremental Semantic Segmentation Without Exemplar Memory
• Split Computing for Complex Object Detectors: Challenges and Preliminary Results
• Regularizing Class-wise Predictions via Self-knowledge Distillation
• A social context-aware graph-based multimodal attentive learning framework for disaster content classification during emergencies: a benchmark dataset and method
• Recent Advances in Deep Learning for Channel Coding: A Survey
• MixKD: Towards Efficient Distillation of Large-scale Language Models
• A Self-Distillation Embedded Supervised Affinity Attention Model for Few-Shot Segmentation
• Continual Learning in Low-rank Orthogonal Subspaces
• Model Distillation for Revenue Optimization: Interpretable Personalized Pricing
• Improving Cross-lingual Information Retrieval on Low-Resource Languages via Optimal Transport Distillation
• ClST: A Convolutional Transformer Framework for Automatic Modulation Recognition by Knowledge Distillation
• Non-autoregressive Transformer by Position Learning
• ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions
• Glance and Focus: a Dynamic Approach to Reducing Spatial Redundancy in Image Classification
• Exploiting Local Structures with the Kronecker Layer in Convolutional Networks
• Multi-Branch Mutual-Distillation Transformer for EEG-Based Seizure Subtype Classification
• Avoiding Your Teacher's Mistakes: Training Neural Networks with Controlled Weak Supervision
• The Impact of the Mini-batch Size on the Variance of Gradients in Stochastic Gradient Descent
• Transfer Learning for Speech and Language Processing
• Improved Trainable Calibration Method for Neural Networks on Medical Imaging Classification
• Data-Driven Sparse Structure Selection for Deep Neural Networks
• Source Free Domain Adaptation with Image Translation
• Security Matters: A Survey on Adversarial Machine Learning
• TwinBERT: Distilling Knowledge to Twin-Structured BERT Models for Efficient Retrieval
• Rain O'er Me: Synthesizing real rain to derain with data distillation
• Embedded Self-Distillation in Compact Multi-Branch Ensemble Network for Remote Sensing Scene Classification
• Few-Shot Stance Detection via Target-Aware Prompt Distillation
• MiniVLM: A Smaller and Faster Vision-Language Model
• Towards Battery-Free Machine Learning and Inference in Underwater Environments
• Fairness via Representation Neutralization
• Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
• FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning
• Non-autoregressive Transformer-based End-to-end ASR using BERT
• Towards Understanding Label Smoothing
• Advancing bioinformatics with large language models: components, applications and perspectives
• Log-DenseNet: How to Sparsify a DenseNet
• IDK Cascades: Fast Deep Learning by Learning not to Overthink
• A survey on Adversarial Recommender Systems: from Attack/Defense strategies to Generative Adversarial Networks
• Class-incremental Learning via Deep Model Consolidation
• Mix&Match - Agent Curricula for Reinforcement Learning
• FeatureNeRF: Learning Generalizable NeRFs by Distilling Foundation Models
• Neural Compatibility Modeling with Attentive Knowledge Distillation
• Rethinking Weight Decay For Efficient Neural Network Pruning
• BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics
• Towards Transmission-Friendly and Robust CNN Models over Cloud and Device
• A Systematic Survey of Deep Learning-based Single-Image Super-Resolution
• Multi-Granularity Regularized Re-Balancing for Class Incremental Learning
• Large-Scale Generative Data-Free Distillation
• Adversarial Robustness via Fisher-Rao Regularization
• Meta-Consolidation for Continual Learning
• Throughput Optimizations for FPGA-based Deep Neural Network Inference
• Kernel Based Progressive Distillation for Adder Neural Networks
• Colorful Image Colorization
• CNN Filter DB: An Empirical Investigation of Trained Convolutional Filters
• Distillation $\approx$ Early Stopping? Harvesting Dark Knowledge Utilizing Anisotropic Information Retrieval For Overparameterized Neural Network
• ReachNN: Reachability Analysis of Neural-Network Controlled Systems
• Increasing the Confidence of Deep Neural Networks by Coverage Analysis
• Noisy Machines: Understanding Noisy Neural Networks and Enhancing Robustness to Analog Hardware Errors Using Distillation
• Low-Resolution Object Recognition with Cross-Resolution Relational Contrastive Distillation
• UR2KiD: Unifying Retrieval, Keypoint Detection, and Keypoint Description without Local Correspondence Supervision
• Channel Compression: Rethinking Information Redundancy among Channels in CNN Architecture
• On the Effectiveness of Low-Rank Matrix Factorization for LSTM Model Compression
• Is Retriever Merely an Approximator of Reader?
• Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation
• Deep Co-Training for Semi-Supervised Image Recognition
• Personalized Federated Learning with First Order Model Optimization
• MoËT: Mixture of Expert Trees and its Application to Verifiable Reinforcement Learning
• Blockwisely Supervised Neural Architecture Search with Knowledge Distillation
• Model compression as constrained optimization, with application to neural nets. Part I: general framework
• SoftHebb: Bayesian Inference in Unsupervised Hebbian Soft Winner-Take-All Networks
• Towards Data Distillation for End-to-end Spoken Conversational Question Answering
• ShrinkTeaNet: Million-scale Lightweight Face Recognition via Shrinking Teacher-Student Networks
• TRANS-BLSTM: Transformer with Bidirectional LSTM for Language Understanding
• Training Shallow and Thin Networks for Acceleration via Knowledge Distillation with Conditional Adversarial Networks
• The Sound Demixing Challenge 2023 $\unicode{x2013}$ Music Demixing Track
• SPIGAN: Privileged Adversarial Learning from Simulation
• Is Label Smoothing Truly Incompatible with Knowledge Distillation: An Empirical Study
• Exploration of Class Center for Fine-Grained Visual Classification
• Distilling Object Detectors via Decoupled Features
• Scalable Transfer Learning with Expert Models
• Survey on Hand Gesture Recognition from Visual Input
• Robust Saliency-Aware Distillation for Few-shot Fine-grained Visual Recognition
• Deep Learning For Face Recognition: A Critical Analysis
• Knowledge Distillation Methods for Efficient Unsupervised Adaptation Across Multiple Domains
• Recent Advances in Deep Learning for Object Detection
• Operational Calibration: Debugging Confidence Errors for DNNs in the Field
• The Threat of Adversarial Attacks on Machine Learning in Network Security -- A Survey
• Minimum Class Confusion for Versatile Domain Adaptation
• A Large RGB-D Dataset for Semi-supervised Monocular Depth Estimation
• COVID-MobileXpert: On-Device COVID-19 Patient Triage and Follow-up using Chest X-rays
• Sub-Band Knowledge Distillation Framework for Speech Enhancement
• A Survey on Deep Learning Architectures for Image-based Depth Reconstruction
• Knowledge Enhanced Pretrained Language Models: A Compreshensive Survey
• CHIP: CHannel Independence-based Pruning for Compact Neural Networks
• FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusion
• Interpreting Deep Learning Models in Natural Language Processing: A Review
• RATT: Recurrent Attention to Transient Tasks for Continual Image Captioning
• Federated Model Distillation with Noise-Free Differential Privacy
• Defense against Universal Adversarial Perturbations
• Improving Multi-Modal Learning with Uni-Modal Teachers
• Task dependent Deep LDA pruning of neural networks
• AdaBERT: Task-Adaptive BERT Compression with Differentiable Neural Architecture Search
• HR-Depth: High Resolution Self-Supervised Monocular Depth Estimation
• Opportunities and Challenges of Deep Learning Methods for Electrocardiogram Data: A Systematic Review
• Mutual Adversarial Training: Learning together is better than going alone
• BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining
• Inference skipping for more efficient real-time speech enhancement with parallel RNNs
• CharacterBERT and Self-Teaching for Improving the Robustness of Dense Retrievers on Queries with Typos
• Lifelong Object Detection
• Hierarchical Autoregressive Image Models with Auxiliary Decoders
• When Ensembling Smaller Models is More Efficient than Single Large Models
• Learning from 2D: Contrastive Pixel-to-Point Knowledge Transfer for 3D Pretraining
• Snorkel DryBell: A Case Study in Deploying Weak Supervision at Industrial Scale
• Joint Multi-Dimension Pruning via Numerical Gradient Update
• DeepReDuce: ReLU Reduction for Fast Private Inference
• Dual-mode ASR: Unify and Improve Streaming ASR with Full-context Modeling
• Born-Again Tree Ensembles
• EEC: Learning to Encode and Regenerate Images for Continual Learning
• DNNShifter: An Efficient DNN Pruning System for Edge Computing
• Teacher-Student chain for efficient semi-supervised histology image classification
• Domain Adaptation without Source Data
• Effectiveness of Distillation Attack and Countermeasure on Neural Network Watermarking
• Vertical Federated Knowledge Transfer via Representation Distillation for Healthcare Collaboration Networks
• Rethinking Feature Distribution for Loss Functions in Image Classification
• Knowledge Distillation Based Semantic Communications For Multiple Users
• Self-supervised visual learning in the low-data regime: a comparative evaluation
• Learning to Iteratively Solve Routing Problems with Dual-Aspect Collaborative Transformer
• Learning Lightweight Pedestrian Detector with Hierarchical Knowledge Distillation
• StyleGAN2 Distillation for Feed-forward Image Manipulation
• AutoSOS: Towards Multi-UAV Systems Supporting Maritime Search and Rescue with Lightweight AI and Edge Computing
• Binarized Collaborative Filtering with Distilling Graph Convolutional Networks
• DiPair: Fast and Accurate Distillation for Trillion-Scale Text Matching and Pair Modeling
• A Survey of Black-Box Adversarial Attacks on Computer Vision Models
• CrisisBERT: a Robust Transformer for Crisis Classification and Contextual Crisis Embedding
• FEED: Feature-level Ensemble for Knowledge Distillation
• Intriguing Properties of Adversarial Examples
• DUT-LFSaliency: Versatile Dataset and Light Field-to-RGB Saliency Detection
• Cross Modal Distillation for Supervision Transfer
• A Survey of Deep Learning Video Super-Resolution
• Capturing the physics of MaNGA galaxies with self-supervised Machine Learning
• Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration
• Optimal Subarchitecture Extraction For BERT
• Adaptive Regularization of Labels
• Action and Perception as Divergence Minimization
• Vision Transformers for Small Histological Datasets Learned through Knowledge Distillation
• Crowd Localization from Gaussian Mixture Scoped Knowledge and Scoped Teacher
• Progressive Feature Alignment for Unsupervised Domain Adaptation
• Counterfactual VQA: A Cause-Effect Look at Language Bias
• Probabilistic Deep Learning to Quantify Uncertainty in Air Quality Forecasting
• Role of Data Augmentation Strategies in Knowledge Distillation for Wearable Sensor Data
• FS-BAN: Born-Again Networks for Domain Generalization Few-Shot Classification
• VOLO: Vision Outlooker for Visual Recognition
• Knowledge Distillation from Internal Representations
• Dataset Inference: Ownership Resolution in Machine Learning
• SAGE: A Split-Architecture Methodology for Efficient End-to-End Autonomous Vehicle Control
• A Comprehensive Survey of Multilingual Neural Machine Translation
• Distillation-based fabric anomaly detection
• Deep LSTM for Large Vocabulary Continuous Speech Recognition
• StackRec: Efficient Training of Very Deep Sequential Recommender Models by Iterative Stacking
• Locally Free Weight Sharing for Network Width Search
• Addressing Missing Labels in Large-Scale Sound Event Recognition Using a Teacher-Student Framework With Loss Masking
• Teacher-Student Framework Enhanced Multi-domain Dialogue Generation
• Active Long Term Memory Networks
• Dataset Distillation
• Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecture
• What does it mean to understand a neural network?
• Unsupervised Person Re-identification via Softened Similarity Learning
• RAIN: RegulArization on Input and Network for Black-Box Domain Adaptation
• Inverse Image Frequency for Long-tailed Image Recognition
• Multimodal Contrastive Learning via Uni-Modal Coding and Cross-Modal Prediction for Multimodal Sentiment Analysis
• Attention-Based Depth Distillation with 3D-Aware Positional Encoding for Monocular 3D Object Detection
• Model-Based Planning with Discrete and Continuous Actions
• Learning Interpretable Rules for Scalable Data Representation and Classification
• Self-similarity Prior Distillation for Unsupervised Remote Physiological Measurement
• What is Next when Sequential Prediction Meets Implicitly Hard Interaction?
• Knowledge Concentration: Learning 100K Object Classifiers in a Single CNN
• Explaining Sequence-Level Knowledge Distillation as Data-Augmentation for Neural Machine Translation
• Meta Transition Adaptation for Robust Deep Learning with Noisy Labels
• Towards Effective Low-bitwidth Convolutional Neural Networks
• Deep Neural Network Fingerprinting by Conferrable Adversarial Examples
• GFL: A Decentralized Federated Learning Framework Based On Blockchain
• End-to-End Multi-Task Learning with Attention
• Less is More: Consistent Video Depth Estimation with Masked Frames Modeling
• It's always personal: Using Early Exits for Efficient On-Device CNN Personalisation
• Hybrid Discriminative-Generative Training via Contrastive Learning
• DBP: Discrimination Based Block-Level Pruning for Deep Model Acceleration
• Balanced softmax cross-entropy for incremental learning with and without memory
• Model Slicing for Supporting Complex Analytics with Elastic Inference Cost and Resource Constraints
• Distilling Knowledge from Graph Convolutional Networks
• Radio2Text: Streaming Speech Recognition Using mmWave Radio Signals
• DarkRank: Accelerating Deep Metric Learning via Cross Sample Similarities Transfer
• Towards a Smaller Student: Capacity Dynamic Distillation for Efficient Image Retrieval
• Knowledge Transfer in Multi-Task Deep Reinforcement Learning for Continuous Control
• Accelerate CNNs from Three Dimensions: A Comprehensive Pruning Framework
• The Monkeytyping Solution to the YouTube-8M Video Understanding Challenge
• DeepTwist: Learning Model Compression via Occasional Weight Distortion
• Ekya: Continuous Learning of Video Analytics Models on Edge Compute Servers
• GhostSR: Learning Ghost Features for Efficient Image Super-Resolution
• Network of Experts for Large-Scale Image Categorization
• Transformer to CNN: Label-scarce distillation for efficient text classification
• FedH2L: Federated Learning with Model and Statistical Heterogeneity
• Edge-cloud Collaborative Learning with Federated and Centralized Features
• Towards Efficient Post-training Quantization of Pre-trained Language Models
• Distilled Neural Networks for Efficient Learning to Rank
• Focal and Global Knowledge Distillation for Detectors
• Residual Knowledge Distillation
• You Impress Me: Dialogue Generation via Mutual Persona Perception
• Distilling Knowledge Learned in BERT for Text Generation
• Deep Learning Markov Random Field for Semantic Segmentation
• Learning Compact Recurrent Neural Networks
• Learning to Parse and Translate Improves Neural Machine Translation
• Geometer: Graph Few-Shot Class-Incremental Learning via Prototype Representation
• Knowledge Consistency between Neural Networks and Beyond
• A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challenges
• Scalable and Adaptive Graph Neural Networks with Self-Label-Enhanced training
• Defense against Adversarial Attacks Using High-Level Representation Guided Denoiser
• TexSmart: A Text Understanding System for Fine-Grained NER and Enhanced Semantic Analysis
• Unwrapping The Black Box of Deep ReLU Networks: Interpretability, Diagnostics, and Simplification
• Resource-Efficient Neural Networks for Embedded Systems
• Self-Knowledge Distillation in Natural Language Processing
• Black-Box Ripper: Copying black-box models using generative evolutionary algorithms
• A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenes
• Listwise View Ranking for Image Cropping
• Knowledge Adaptation for Efficient Semantic Segmentation
• Improved Techniques for Training Adaptive Deep Networks
• Large-Scale Domain Adaptation via Teacher-Student Learning
• Reliable Adversarial Distillation with Unreliable Teachers
• Fighting Quantization Bias With Bias
• Computer Vision Model Compression Techniques for Embedded Systems: A Survey
• Scalable Zero-shot Entity Linking with Dense Entity Retrieval
• Private Model Compression via Knowledge Distillation
• Tackling the Unannotated: Scene Graph Generation with Bias-Reduced Models
• Dual Student: Breaking the Limits of the Teacher in Semi-supervised Learning
• Self-supervised Moving Vehicle Tracking with Stereo Sound
• Exploring Advances in Transformers and CNN for Skin Lesion Diagnosis on Small Datasets
• WSOD^2: Learning Bottom-up and Top-down Objectness Distillation for Weakly-supervised Object Detection
• MKD: a Multi-Task Knowledge Distillation Approach for Pretrained Language Models
• Preparing Lessons: Improve Knowledge Distillation with Better Supervision
• Modeling Lost Information in Lossy Image Compression
• Particle streak velocimetry using Ensemble Convolutional Neural Networks
• ZeroQ: A Novel Zero Shot Quantization Framework
• LadaBERT: Lightweight Adaptation of BERT through Hybrid Model Compression
• SE-SSD: Self-Ensembling Single-Stage Object Detector From Point Cloud
• Imitation Attacks and Defenses for Black-box Machine Translation Systems
• Joint Neural Architecture Search and Quantization
• Few-Shot Incremental Learning with Continually Evolved Classifiers
• Semi-Supervised Sequence Modeling with Cross-View Training
• Ranking Distillation: Learning Compact Ranking Models With High Performance for Recommender System
• Effective Use of Synthetic Data for Urban Scene Semantic Segmentation
• CATBERT: Context-Aware Tiny BERT for Detecting Social Engineering Emails
• SeReNe: Sensitivity based Regularization of Neurons for Structured Sparsity in Neural Networks
• MetaLabelNet: Learning to Generate Soft-Labels from Noisy-Labels
• Anchor Pruning for Object Detection
• 3D Human Action Representation Learning via Cross-View Consistency Pursuit
• A Dimensionality Reduction Approach for Convolutional Neural Networks
• OBMO: One Bounding Box Multiple Objects for Monocular 3D Object Detection
• Audit to Forget: A Unified Method to Revoke Patients' Private Data in Intelligent Healthcare
• Communication Resources Constrained Hierarchical Federated Learning for End-to-End Autonomous Driving
• Multi-Label Knowledge Distillation
• ACDC: A Structured Efficient Linear Layer
• Continuous fake media detection: adapting deepfake detectors to new generative techniques
• Undistillable: Making A Nasty Teacher That CANNOT teach students
• Label-guided Attention Distillation for Lane Segmentation
• ProKnow: Process Knowledge for Safety Constrained and Explainable Question Generation for Mental Health Diagnostic Assistance
• BlockCopy: High-Resolution Video Processing with Block-Sparse Feature Propagation and Online Policies
• Binarized Neural Networks on the ImageNet Classification Task
• Weight-dependent Gates for Network Pruning
• Collaborative Deep Reinforcement Learning
• Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression
• Fast ConvNets Using Group-wise Brain Damage
• Cross-Resolution Face Recognition via Prior-Aided Face Hallucination and Residual Knowledge Distillation
• Regression Prior Networks
• Knowledge Amalgamation for Object Detection with Transformers
• Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
• Predicting Events in MOBA Games: Prediction, Attribution, and Evaluation
• Role-Wise Data Augmentation for Knowledge Distillation
• Non-Autoregressive Machine Translation with Auxiliary Regularization
• Semantics and explanation: why counterfactual explanations produce adversarial examples in deep neural networks
• Omni-sourced Webly-supervised Learning for Video Recognition
• Learning Soft Labels via Meta Learning
• Few-Shot Speaker Identification Using Lightweight Prototypical Network with Feature Grouping and Interaction
• Multiple Expert Brainstorming for Domain Adaptive Person Re-identification
• HAWQV3: Dyadic Neural Network Quantization
• Finite size corrections for neural network Gaussian processes
• FRILL: A Non-Semantic Speech Embedding for Mobile Devices
• Tensorial Neural Networks: Generalization of Neural Networks and Application to Model Compression
• SegBlocks: Block-Based Dynamic Resolution Networks for Real-Time Segmentation
• Knapsack Pruning with Inner Distillation
• Does label smoothing mitigate label noise?
• Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation
• Unsupervised Domain Adaption Harnessing Vision-Language Pre-training
• Optimizing speed/accuracy trade-off for person re-identification via knowledge distillation
• Fast On-the-fly Retraining-free Sparsification of Convolutional Neural Networks
• A framework for fostering transparency in shared artificial intelligence models by increasing visibility of contributions
• Data-Free/Data-Sparse Softmax Parameter Estimation with Structured Class Geometries
• Attack and Defense of Dynamic Analysis-Based, Adversarial Neural Malware Classification Models
• Knowledge Infused Learning (K-IL): Towards Deep Incorporation of Knowledge in Deep Learning
• Attended Temperature Scaling: A Practical Approach for Calibrating Deep Neural Networks
• FrankenSplit: Efficient Neural Feature Compression with Shallow Variational Bottleneck Injection for Mobile Edge Computing
• Knowing What, Where and When to Look: Efficient Video Action Modeling with Attention
• Learning Compact Recurrent Neural Networks with Block-Term Tensor Decomposition
• Two-Level Residual Distillation based Triple Network for Incremental Object Detection
• Deep Fried Convnets
• Enhancing the Robustness of Deep Neural Networks by Boundary Conditional GAN
• Learning Distilled Collaboration Graph for Multi-Agent Perception
• TIP: Typifying the Interpretability of Procedures
• Distilling Causal Effect of Data in Class-Incremental Learning
• Scalable and Order-robust Continual Learning with Additive Parameter Decomposition
• xCos: An Explainable Cosine Metric for Face Verification Task
• Universally Slimmable Networks and Improved Training Techniques
• Model Distillation with Knowledge Transfer from Face Classification to Alignment and Verification
• MSD: Multi-Self-Distillation Learning via Multi-classifiers within Deep Neural Networks
• Knowledge Distillation with Adversarial Samples Supporting Decision Boundary
• On the Reproducibility of Neural Network Predictions
• Continual Classification Learning Using Generative Models
• Group channel pruning and spatial attention distilling for object detection
• Cross-X Learning for Fine-Grained Visual Categorization
• Attacking Optical Character Recognition (OCR) Systems with Adversarial Watermarks
• Few Shot Network Compression via Cross Distillation
• Knowledge Cross-Distillation for Membership Privacy
• Diversity Matters When Learning From Ensembles
• Flexible Dataset Distillation: Learn Labels Instead of Images
• Distilling Large Language Models into Tiny and Effective Students using pQRNN
• Mixed Precision Low-bit Quantization of Neural Network Language Models for Speech Recognition
• A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems
• HardCoRe-NAS: Hard Constrained diffeRentiable Neural Architecture Search
• ASiT: Local-Global Audio Spectrogram vIsion Transformer for Event Classification
• Understanding Multi-Turn Toxic Behaviors in Open-Domain Chatbots
• HOPE: Hybrid-granularity Ordinal Prototype Learning for Progression Prediction of Mild Cognitive Impairment
• DDFlow: Learning Optical Flow with Unlabeled Data Distillation
• New Insights on Relieving Task-Recency Bias for Online Class Incremental Learning
• Self-training for Few-shot Transfer Across Extreme Task Differences
• Refining Language Models with Compositional Explanations
• An Adaptive Random Path Selection Approach for Incremental Learning
• Copycat CNN: Are Random Non-Labeled Data Enough to Steal Knowledge from Black-box Models?
• Intermediate Deep Feature Compression: the Next Battlefield of Intelligent Sensing
• Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions
• Entity Linking and Discovery via Arborescence-based Supervised Clustering
• Robust Learning Under Label Noise With Iterative Noise-Filtering
• A Teacher-Student Framework for Semi-supervised Medical Image Segmentation From Mixed Supervision
• Dynamic Hard Pruning of Neural Networks at the Edge of the Internet
• Selfish Sparse RNN Training
• Interpreting Deep Classifier by Visual Distillation of Dark Knowledge
• I-BERT: Integer-only BERT Quantization
• Blockwise Self-Attention for Long Document Understanding
• Exploring Benefits of Transfer Learning in Neural Machine Translation
• Switchable Precision Neural Networks
• Iteratively Training Look-Up Tables for Network Quantization
• Percival: Making In-Browser Perceptual Ad Blocking Practical With Deep Learning
• Small and Practical BERT Models for Sequence Labeling
• FQ-Conv: Fully Quantized Convolution for Efficient and Accurate Inference
• Why distillation helps: a statistical perspective
• Evaluating Deep Graph Neural Networks
• Beyond Voice Activity Detection: Hybrid Audio Segmentation for Direct Speech Translation
• Self-distillation with Batch Knowledge Ensembling Improves ImageNet Classification
• Refine Myself by Teaching Myself: Feature Refinement via Self-Knowledge Distillation
• Energy Efficient Hadamard Neural Networks
• Knowledge Projection for Deep Neural Networks
• Automatic Mixed-Precision Quantization Search of BERT
• Structured Knowledge Distillation for Dense Prediction
• Autonomous Deep Learning: Incremental Learning of Denoising Autoencoder for Evolving Data Streams
• Anonymizing Machine Learning Models
• Bag of Baselines for Multi-objective Joint Neural Architecture Search and Hyperparameter Optimization
• Explaining Knowledge Distillation by Quantifying the Knowledge
• Model Compression with Multi-Task Knowledge Distillation for Web-scale Question Answering System
• Knowledge distillation via adaptive instance normalization
• Post-Training Piecewise Linear Quantization for Deep Neural Networks
• BEBERT: Efficient and Robust Binary Ensemble BERT
• Few Sample Knowledge Distillation for Efficient Network Compression
• Leveraging Ensembles and Self-Supervised Learning for Fully-Unsupervised Person Re-Identification and Text Authorship Attribution
• Self-Knowledge Distillation with Progressive Refinement of Targets
• Few-Shot Class-Incremental Learning
• Regularization Matters: A Nonparametric Perspective on Overparametrized Neural Network
• Assisted Learning: A Framework for Multi-Organization Learning
• Learning at a Glance: Towards Interpretable Data-limited Continual Semantic Segmentation via Semantic-Invariance Modelling
• MMDRFuse: Distilled Mini-Model with Dynamic Refresh for Multi-Modality Image Fusion
• Knowledge Distillation for End-to-End Person Search
• Pre-Trained Models: Past, Present and Future
• Joint framework with deep feature distillation and adaptive focal loss for weakly supervised audio tagging and acoustic event detection
• SeqNAS: Neural Architecture Search for Event Sequence Classification
• Multi-Channel Multi-Domain based Knowledge Distillation Algorithm for Sleep Staging with Single-Channel EEG
• Exact and Consistent Interpretation for Piecewise Linear Neural Networks: A Closed Form Solution
• Towards Training Recurrent Neural Networks for Lifelong Learning
• Synthetic data generation method for data-free knowledge distillation in regression neural networks
• KTAN: Knowledge Transfer Adversarial Network
• Amalgamating Knowledge towards Comprehensive Classification
• MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers
• Distilling Word Embeddings: An Encoding Approach
• Revisiting Knowledge Distillation via Label Smoothing Regularization
• Learning to Estimate Without Bias
• Continual 3D Convolutional Neural Networks for Real-time Processing of Videos
• Textual Data Augmentation for Patient Outcomes Prediction
• Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyond
• Knowledge distillation using unlabeled mismatched images
• Confidence from Invariance to Image Transformations
• Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks
• Federated Learning for Energy Constrained IoT devices: A systematic mapping study
• RGB-based 3D Hand Pose Estimation via Privileged Learning with Depth Images
• Neural Document Summarization by Jointly Learning to Score and Select Sentences
• One Person, One Model, One World: Learning Continual User Representation without Forgetting
• Extract-Transform-Load for Video Streams
• Fine-Pruning: Joint Fine-Tuning and Compression of a Convolutional Network with Bayesian Optimization
• Accuracy enhancement method for speech emotion recognition from spectrogram using temporal frequency correlation and positional information learning through knowledge transfer
• Towards Black-box Iterative Machine Teaching
• A Predictive Approach To Enhance Time-Series Forecasting
• Single Shot Structured Pruning Before Training
• Representation Selective Self-distillation and wav2vec 2.0 Feature Exploration for Spoof-aware Speaker Verification
• Foundation Models and Transformers for Anomaly Detection: A Survey
• Dense Neural Network Based Arrhythmia Classification on Low-cost and Low-compute Micro-controller
• Detecting Human-Object Interactions with Action Co-occurrence Priors
• Constructing Deep Neural Networks by Bayesian Network Structure Learning
• DeepObfuscation: Securing the Structure of Convolutional Neural Networks via Knowledge Distillation
• Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noise
• Learning Efficient Detector with Semi-supervised Adaptive Distillation
• Natural and Adversarial Error Detection using Invariance to Image Transformations
• SBNet: Sparse Blocks Network for Fast Inference
• Emotion Recognition in Speech using Cross-Modal Transfer in the Wild
• An Ensemble of Knowledge Sharing Models for Dynamic Hand Gesture Recognition
• Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structure
• Learning Anytime Predictions in Neural Networks via Adaptive Loss Balancing
• Heuristic-Guided Reinforcement Learning
• Rethinking Bottleneck Structure for Efficient Mobile Network Design
• SCAN: A Scalable Neural Networks Framework Towards Compact and Efficient Models
• Leveraging Weakly Supervised Data to Improve End-to-End Speech-to-Text Translation
• Convolutional Neural Network Pruning with Structural Redundancy Reduction
• Generalization bounds via distillation
• ProSelfLC: Progressive Self Label Correction for Training Robust Deep Neural Networks
• MEAL: Multi-Model Ensemble via Adversarial Learning
• Universal Perturbation Attack Against Image Retrieval
• Language Conditioned Spatial Relation Reasoning for 3D Object Grounding
• Multitask Emotion Recognition with Incomplete Labels
• Ensemble Robustness and Generalization of Stochastic Deep Learning Algorithms
• A Neural Span-Based Continual Named Entity Recognition Model
• Compact Trilinear Interaction for Visual Question Answering
• A Survey on Resilient Machine Learning
• Knowledge Transfer Pre-training
• Multi-Scale and Multi-Layer Contrastive Learning for Domain Generalization
• Generative Low-bitwidth Data Free Quantization
• An Experimental Study of the Impact of Pre-training on the Pruning of a Convolutional Neural Network
• FedLoc: Federated Learning Framework for Data-Driven Cooperative Localization and Location Data Processing
• KDk: A Defense Mechanism Against Label Inference Attacks in Vertical Federated Learning
• The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding
• AdderNet: Do We Really Need Multiplications in Deep Learning?
• A First Look at Class Incremental Learning in Deep Learning Mobile Traffic Classification
• Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillation
• Robust Multi-Modal Sensor Fusion: An Adversarial Approach
• Bringing Giant Neural Networks Down to Earth with Unlabeled Data
• Online Continual Learning via the Knowledge Invariant and Spread-out Properties
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks
• Zero-Shot Knowledge Distillation from a Decision-Based Black-Box Model
• SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies
• Transferring Inductive Biases through Knowledge Distillation
• Parameterized Hypercomplex Graph Neural Networks for Graph Classification
• Token-Level Ensemble Distillation for Grapheme-to-Phoneme Conversion
• Ternary Compression for Communication-Efficient Federated Learning
• KDLSQ-BERT: A Quantized Bert Combining Knowledge Distillation with Learned Step Size Quantization
• Class-Incremental Few-Shot Object Detection
• Multiple Domain Experts Collaborative Learning: Multi-Source Domain Generalization For Person Re-Identification
• Maximizing Discrimination Capability of Knowledge Distillation with Energy Function
• Rectified Decision Trees: Towards Interpretability, Compression and Empirical Soundness
• CoCoPIE: Making Mobile AI Sweet As PIE --Compression-Compilation Co-Design Goes a Long Way
• Security and Privacy for Artificial Intelligence: Opportunities and Challenges
• ACE: Adapting to Changing Environments for Semantic Segmentation
• Attention-Guided Answer Distillation for Machine Reading Comprehension
• Finding Nano-Ötzi: Semi-Supervised Volume Visualization for Cryo-Electron Tomography
• Federated Stain Normalization for Computational Pathology
• MobileStyleGAN: A Lightweight Convolutional Neural Network for High-Fidelity Image Synthesis
• Towards Trustworthy Dataset Distillation
• Learning Intra-Batch Connections for Deep Metric Learning
• Learning Adaptive Classifiers Synthesis for Generalized Few-Shot Learning
• Continual Learning in Neural Networks
• Effective End-to-End Vision Language Pretraining with Semantic Visual Loss
• Teacher-Student Compression with Generative Adversarial Networks
• Learnable Embedding Space for Efficient Neural Architecture Compression
• Dropout with Expectation-linear Regularization
• LibFewShot: A Comprehensive Library for Few-shot Learning
• Learning to Steer by Mimicking Features from Heterogeneous Auxiliary Networks
• Fast, Accurate, and Simple Models for Tabular Data via Augmented Distillation
• Continual Learning in Recurrent Neural Networks
• Few-Shot Self Reminder to Overcome Catastrophic Forgetting
• Densely Connected Search Space for More Flexible Neural Architecture Search
• Neural networks on microcontrollers: saving memory at inference via operator reordering
• Fast and Accurate Single-Image Depth Estimation on Mobile Devices, Mobile AI 2021 Challenge: Report
• ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training
• AdaNet: A Scalable and Flexible Framework for Automatically Learning Ensembles
• Temperature check: theory and practice for training models with softmax-cross-entropy losses
• BlockSwap: Fisher-guided Block Substitution for Network Compression on a Budget
• Knowledge Squeezed Adversarial Network Compression
• Causal Interpretability for Machine Learning -- Problems, Methods and Evaluation
• Source-Free Domain Adaptation for Semantic Segmentation
• SignalNet: A Low Resolution Sinusoid Decomposition and Estimation Network
• Search for Better Students to Learn Distilled Knowledge
• Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Models
• A Secure Mobile Authentication Alternative to Biometrics
• From the Lab to the Wild: Affect Modeling via Privileged Information
• Adversarial Detection and Correction by Matching Prediction Distributions
• Progressive Domain Expansion Network for Single Domain Generalization
• Self-informed neural network structure learning
• Human-Inspired Multi-Agent Navigation using Knowledge Distillation
• Towards a General Deep Feature Extractor for Facial Expression Recognition
• A Survey of AI-Related Cyber Security Risks and Countermeasures in Mobility-as-a-Service
• An Empirical Analysis of the Impact of Data Augmentation on Knowledge Distillation
• Hyperparameter-Free Out-of-Distribution Detection Using Softmax of Scaled Cosine Similarity
• Understanding the role of importance weighting for deep learning
• Orchestrating the Development Lifecycle of Machine Learning-Based IoT Applications: A Taxonomy and Survey
• Multi-Target Domain Adaptation with Collaborative Consistency Learning
• Style Example-Guided Text Generation using Generative Adversarial Transformers
• Towards Low-Latency Energy-Efficient Deep SNNs via Attention-Guided Compression
• An Embarrassingly Simple Approach for Knowledge Distillation
• AdvSPADE: Realistic Unrestricted Attacks for Semantic Segmentation
• More Grounded Image Captioning by Distilling Image-Text Matching Model
• Single-/Multi-Source Cross-Lingual NER via Teacher-Student Learning on Unlabeled Data in Target Language
• Breast Mass Detection with Faster R-CNN: On the Feasibility of Learning from Noisy Annotations
• Pipeline Parallelism for Inference on Heterogeneous Edge Computing
• Narrowing the Gap: Improved Detector Training with Noisy Location Annotations
• Global-Local Bidirectional Reasoning for Unsupervised Representation Learning of 3D Point Clouds
• Representation Stability as a Regularizer for Improved Text Analytics Transfer Learning
• A New Ensemble Learning Framework for 3D Biomedical Image Segmentation
• FedDCT: Federated Learning of Large Convolutional Neural Networks on Resource Constrained Devices using Divide and Collaborative Training
• FeTrIL: Feature Translation for Exemplar-Free Class-Incremental Learning
• Analysis of Deep Networks for Monocular Depth Estimation Through Adversarial Attacks with Proposal of a Defense Method
• Learning Interpretable Models with Causal Guarantees
• Local Black-box Adversarial Attacks: A Query Efficient Approach
• LIT: Block-wise Intermediate Representation Training for Model Compression
• Naive-Student: Leveraging Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation
• Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis
• Learnable Boundary Guided Adversarial Training
• Multi-View Attention Transfer for Efficient Speech Enhancement
• A Comprehensive Overhaul of Feature Distillation
• MimicDet: Bridging the Gap Between One-Stage and Two-Stage Object Detection
• Meta-Learning across Meta-Tasks for Few-Shot Learning
• Semantic Edge Computing and Semantic Communications in 6G Networks: A Unifying Survey and Research Challenges
• Modularized Morphing of Neural Networks
• MECKD: Deep Learning-Based Fall Detection in Multilayer Mobile Edge Computing With Knowledge Distillation
• A Practical Incremental Method to Train Deep CTR Models
• Cloth-Changing Person Re-identification from A Single Image with Gait Prediction and Regularization
• Anytime Neural Prediction via Slicing Networks Vertically
• Paint and Distill: Boosting 3D Object Detection with Semantic Passing Network
• Deep cross-modality (MR-CT) educed distillation learning for cone beam CT lung tumor segmentation
• Reflective-Net: Learning from Explanations
• Finding Fast Transformers: One-Shot Neural Architecture Search by Component Composition
• LAFFNet: A Lightweight Adaptive Feature Fusion Network for Underwater Image Enhancement
• Robustness-Reinforced Knowledge Distillation with Correlation Distance and Network Pruning
• Enhancing Consistency and Mitigating Bias: A Data Replay Approach for Incremental Learning
• Walsh-domain Neural Network for Power Amplifier Behavioral Modelling and Digital Predistortion
• Compressing Deep Image Super-resolution Models
• Acoustic scene classification using teacher-student learning with soft-labels
• Improving Regression Performance with Distributional Losses
• PAMS: Quantized Super-Resolution via Parameterized Max Scale
• Improving Multi-Person Pose Estimation using Label Correction
• Semi-Autoregressive Neural Machine Translation
• Noisy Self-Knowledge Distillation for Text Summarization
• Person Search by Multi-Scale Matching
• Distilling a Deep Neural Network into a Takagi-Sugeno-Kang Fuzzy Inference System
• Cascaded Text Generation with Markov Transformers
• All You Need is a Few Shifts: Designing Efficient Convolutional Neural Networks for Image Classification
• More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval
• Learning the Relation between Similarity Loss and Clustering Loss in Self-Supervised Learning
• ATHENA: A Framework based on Diverse Weak Defenses for Building Adversarial Defense
• Efficient and Private Federated Learning with Partially Trainable Networks
• MEGEX: Data-Free Model Extraction Attack against Gradient-Based Explainable AI
• Deep Transfer Learning for Classification of Variable Sources
• Distillation Techniques for Pseudo-rehearsal Based Incremental Learning
• Unpaired Multi-modal Segmentation via Knowledge Distillation
• Adversarial Examples on Object Recognition: A Comprehensive Survey
• Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models
• A Fourier-based Framework for Domain Generalization
• Rethinking embedding coupling in pre-trained language models
• Reducing the Teacher-Student Gap via Spherical Knowledge Disitllation
• Retrieving Sequential Information for Non-Autoregressive Neural Machine Translation
• Activity-aware Human Mobility Prediction with Hierarchical Graph Attention Recurrent Network
• LTD: Low Temperature Distillation for Gradient Masking-free Adversarial Training
• A Thorough Comparison Study on Adversarial Attacks and Defenses for Common Thorax Disease Classification in Chest X-rays
• Depth separation for reduced deep networks in nonlinear model reduction: Distilling shock waves in nonlinear hyperbolic problems
• Improving the Interpretability of Deep Neural Networks with Knowledge Distillation
• Multimodal Side-Tuning for Document Classification
• FxP-QNet: A Post-Training Quantizer for the Design of Mixed Low-Precision DNNs with Dynamic Fixed-Point Representation
• Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation
• A Meta-Evaluation of Faithfulness Metrics for Long-Form Hospital-Course Summarization
• OffRoadTranSeg: Semi-Supervised Segmentation using Transformers on OffRoad environments
• Neural Network Pruning with Residual-Connections and Limited-Data
• SING: Symbol-to-Instrument Neural Generator
• Automated Design Space Exploration for optimised Deployment of DNN on Arm Cortex-A CPUs
• When Does Preconditioning Help or Hurt Generalization?
• Ranking and Tuning Pre-trained Models: A New Paradigm for Exploiting Model Hubs
• Compressing Neural Language Models by Sparse Word Representations
• Pruning a restricted Boltzmann machine for quantum state reconstruction
• Distilling Knowledge via Knowledge Review
• Hint-Based Training for Non-Autoregressive Machine Translation
• Dataset balancing can hurt model performance
• GBDT-MO: Gradient Boosted Decision Trees for Multiple Outputs
• Continual Learning in Human Activity Recognition: an Empirical Analysis of Regularization
• Joint Contrastive Learning with Infinite Possibilities
• The Training Process of Many Deep Networks Explores the Same Low-Dimensional Manifold
• Compacting Neural Network Classifiers via Dropout Training
• Parser-Free Virtual Try-on via Distilling Appearance Flows
• An Efficient Transformer Decoder with Compressed Sub-layers
• Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learning
• Incremental Learning with Maximum Entropy Regularization: Rethinking Forgetting and Intransigence
• Training Networks in Null Space of Feature Covariance for Continual Learning
• Angular Visual Hardness
• Fast Vocabulary Transfer for Language Model Compression
• Empirical Analysis of Training Strategies of Transformer-based Japanese Chit-chat Systems
• ACT-Net: Asymmetric Co-Teacher Network for Semi-supervised Memory-efficient Medical Image Segmentation
• Spatially Adaptive Inference with Stochastic Feature Sampling and Interpolation
• Listen to Look: Action Recognition by Previewing Audio
• Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting
• KroneckerBERT: Learning Kronecker Decomposition for Pre-trained Language Models via Knowledge Distillation
• Collaborative Intelligent Cross-Camera Video Analytics at Edge: Opportunities and Challenges
• Collaboration and Transition: Distilling Item Transitions into Multi-Query Self-Attention for Sequential Recommendation
• Temporally Distributed Networks for Fast Video Semantic Segmentation
• Generalized Continual Zero-Shot Learning
• In-Ear-Voice: Towards Milli-Watt Audio Enhancement With Bone-Conduction Microphones for In-Ear Sensing Platforms
• Deep Face Recognition Model Compression via Knowledge Transfer and Distillation
• Hard-Attention for Scalable Image Classification
• Rethinking and Improving the Robustness of Image Style Transfer
• Continual Policy Distillation of Reinforcement Learning-based Controllers for Soft Robotic In-Hand Manipulation
• Identifying the Defective: Detecting Damaged Grains for Cereal Appearance Inspection
• Robustness of Accuracy Metric and its Inspirations in Learning with Noisy Labels
• Transient Non-Stationarity and Generalisation in Deep Reinforcement Learning
• Training Group Orthogonal Neural Networks with Privileged Information
• Deep Q-Networks for Accelerating the Training of Deep Neural Networks
• Multi-Attention Based Ultra Lightweight Image Super-Resolution
• Weight Normalization based Quantization for Deep Neural Network Compression
• Fully Non-autoregressive Neural Machine Translation: Tricks of the Trade
• PyNET-QxQ: An Efficient PyNET Variant for QxQ Bayer Pattern Demosaicing in CMOS Image Sensors
• MiCE: Mixture of Contrastive Experts for Unsupervised Image Clustering
• Being Bayesian about Categorical Probability
• Precise neural network computation with imprecise analog devices
• Cyclic Differentiable Architecture Search
• Customizing Student Networks From Heterogeneous Teachers via Adaptive Knowledge Amalgamation
• CO2: Consistent Contrast for Unsupervised Visual Representation Learning
• Iterative Graph Self-Distillation
• Ensemble Knowledge Distillation for Learning Improved and Efficient Networks
• From Federated Learning to Federated Neural Architecture Search: A Survey
• SF-UDA${}^{3D}$: Source-Free Unsupervised Domain Adaptation for LiDAR-Based 3D Object Detection
• Balanced Knowledge Distillation for Long-tailed Learning
• SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption
• On-FPGA Training with Ultra Memory Reduction: A Low-Precision Tensor Method
• An Incremental Learning framework for Large-scale CTR Prediction
• Student Becoming the Master: Knowledge Amalgamation for Joint Scene Parsing, Depth Estimation, and More
• Variational Prototype Replays for Continual Learning
• Run Away From your Teacher: Understanding BYOL by a Novel Self-Supervised Approach
• Multi-task Learning for Continuous Control
• DMCL: Distillation Multiple Choice Learning for Multimodal Action Recognition
• ContCap: A scalable framework for continual image captioning
• Patch Slimming for Efficient Vision Transformers
• OMNIA Faster R-CNN: Detection in the wild through dataset merging and soft distillation
• Unsupervised Image Captioning
• Progressively Stacking 2.0: A Multi-stage Layerwise Training Method for BERT Training Speedup
• Incremental Learning In Online Scenario
• Soft Prompt Decoding for Multilingual Dense Retrieval
• Collapsible Linear Blocks for Super-Efficient Super Resolution
• Continual Learning for Monolingual End-to-End Automatic Speech Recognition
• TernaryBERT: Distillation-aware Ultra-low Bit BERT
• DC-BERT: Decoupling Question and Document for Efficient Contextual Encoding
• Segment Anything Model is a Good Teacher for Local Feature Learning
• DOCTOR: A Simple Method for Detecting Misclassification Errors
• Learning Deep Representations with Probabilistic Knowledge Transfer
• CATFace: Cross-Attribute-Guided Transformer with Self-Attention Distillation for Low-Quality Face Recognition
• Interpreting Deep Neural Networks Through Variable Importance
• Fully-adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classification
• Building Robust Deep Neural Networks for Road Sign Detection
• Nonparametric Neural Networks
• How intelligent are convolutional neural networks?
• Share your Model instead of your Data: Privacy Preserving Mimic Learning for Ranking
• Improving Neural Architecture Search Image Classifiers via Ensemble Learning
• GASL: Guided Attention for Sparsity Learning in Deep Neural Networks
• Filter Grafting for Deep Neural Networks
• In Defense of the Triplet Loss Again: Learning Robust Person Re-Identification with Fast Approximated Triplet Loss and Label Distillation
• CUP: Cluster Pruning for Compressing Deep Neural Networks
• Graph Representation Learning via Multi-task Knowledge Distillation
• Multi-Frame to Single-Frame: Knowledge Distillation for 3D Object Detection
• Collaborative Distillation in the Parameter and Spectrum Domains for Video Action Recognition
• RGB-IR Cross-modality Person ReID based on Teacher-Student GAN Model
• Soft Gradient Boosting Machine
• DeFormer: Decomposing Pre-trained Transformers for Faster Question Answering
• LightPAFF: A Two-Stage Distillation Framework for Pre-training and Fine-tuning
• Inter-Region Affinity Distillation for Road Marking Segmentation
• Half-Real Half-Fake Distillation for Class-Incremental Semantic Segmentation
• Learning Scene Structure Guidance via Cross-Task Knowledge Transfer for Single Depth Super-Resolution
• tf.data: A Machine Learning Data Processing Framework
• DistFL: Distribution-aware Federated Learning for Mobile Scenarios
• The More, the Better? A Study on Collaborative Machine Learning for DGA Detection
• Class-Distribution-Aware Calibration for Long-Tailed Visual Recognition
• Semi-Supervised Domain Generalizable Person Re-Identification
• Sensor-Augmented Egocentric-Video Captioning with Dynamic Modal Attention
• Affect Expression Behaviour Analysis in the Wild using Consensual Collaborative Training
• Consensual Collaborative Training And Knowledge Distillation Based Facial Expression Recognition Under Noisy Annotations
• Scalable Transformers for Neural Machine Translation
• XtremeDistilTransformers: Task Transfer for Task-agnostic Distillation
• Early-stopped neural networks are consistent
• A Generalized Zero-Shot Quantization of Deep Convolutional Neural Networks via Learned Weights Statistics
• Rationalizing Predictions by Adversarial Information Calibration
• On the Importance of Denoising when Learning to Compress Images
• An investigation of structures responsible for gender bias in BERT and DistilBERT
• Label-Efficient Semantic Segmentation of LiDAR Point Clouds in Adverse Weather Conditions
• Cyber Attacks Prevention Towards Prosumer-based EV Charging Stations: An Edge-assisted Federated Prototype Knowledge Distillation Approach
• Knowledge Distillation via Weighted Ensemble of Teaching Assistants
• Does Simultaneous Speech Translation need Simultaneous Models?
• Augmenting semantic lexicons using word embeddings and transfer learning
• Learning from Temporal Gradient for Semi-supervised Action Recognition
• Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoods
• Model-Based Opponent Modeling
• It Takes Two to Tango: Mixup for Deep Metric Learning
• MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space
• Boosting Light-Weight Depth Estimation Via Knowledge Distillation
• Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning
• Learning Frequency Domain Approximation for Binary Neural Networks
• Parallel Training of Deep Networks with Local Updates
• Deep Serial Number: Computational Watermarking for DNN Intellectual Property Protection
• On the Exploration of Incremental Learning for Fine-grained Image Retrieval
• Load What You Need: Smaller Versions of Multilingual BERT
• Length-Adaptive Transformer: Train Once with Length Drop, Use Anytime with Search
• Lifelong Graph Learning
• PERF-Net: Pose Empowered RGB-Flow Net
• Learning from Multiple Datasets with Heterogeneous and Partial Labels for Universal Lesion Detection in CT
• Growing Efficient Deep Networks by Structured Continuous Sparsification
• Multi-label Contrastive Predictive Coding
• Classification Under Misspecification: Halfspaces, Generalized Linear Models, and Connections to Evolvability
• Towards Practical Lottery Ticket Hypothesis for Adversarial Training
• Efficient Semantic Video Segmentation with Per-frame Inference
• Any-Precision Deep Neural Networks
• Distilling BERT into Simple Neural Networks with Unlabeled Transfer Data
• Effective Training of Convolutional Neural Networks with Low-bitwidth Weights and Activations
• Personal VAD: Speaker-Conditioned Voice Activity Detection
• Semantic-Aware Knowledge Preservation for Zero-Shot Sketch-Based Image Retrieval
• TKD: Temporal Knowledge Distillation for Active Perception
• Transfer Meets Hybrid: A Synthetic Approach for Cross-Domain Collaborative Filtering with Text
• SwishNet: A Fast Convolutional Neural Network for Speech, Music and Noise Classification and Segmentation
• Selective Distillation of Weakly Annotated GTD for Vision-based Slab Identification System
• Improved training for online end-to-end speech recognition systems
• Derivative Manipulation for General Example Weighting
• Deep Neural Networks for Choice Analysis: A Statistical Learning Theory Perspective
• Leveraging Bottom-Up and Top-Down Attention for Few-Shot Object Detection
• Towards Cross-modality Medical Image Segmentation with Online Mutual Knowledge Distillation
• Enhancing Explainability of Neural Networks through Architecture Constraints
• Progressive Distillation Based on Masked Generation Feature Method for Knowledge Graph Completion
• Distilling Missing Modality Knowledge from Ultrasound for Endometriosis Diagnosis with Magnetic Resonance Images
• UNIDEAL: Curriculum Knowledge Distillation Federated Learning
• ZPD Teaching Strategies for Deep Reinforcement Learning from Demonstrations
• An Empirical Study of Efficient ASR Rescoring with Transformers
• Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation
• Boosting Binary Masks for Multi-Domain Learning through Affine Transformations
• All-Transfer Learning for Deep Neural Networks and its Application to Sepsis Classification
• Fast End-to-End Speech Recognition via Non-Autoregressive Models and Cross-Modal Knowledge Transferring from BERT
• Learning Metrics from Teachers: Compact Networks for Image Embedding
• Leveraging Angular Distributions for Improved Knowledge Distillation
• Joint User Association and Power Allocation in Heterogeneous Ultra Dense Network via Semi-Supervised Representation Learning
• Transferring Knowledge Distillation for Multilingual Social Event Detection
• Network Quantization with Element-wise Gradient Scaling
• On Universal Black-Box Domain Adaptation
• Thief, Beware of What Get You There: Towards Understanding Model Extraction Attack
• Towards Adversarial-Resilient Deep Neural Networks for False Data Injection Attack Detection in Power Grids
• Adaptive Consistency Regularization for Semi-Supervised Transfer Learning
• Toward Compact Deep Neural Networks via Energy-Aware Pruning
• Data Distillation for Text Classification
• Continual Learning using a Bayesian Nonparametric Dictionary of Weight Factors
• Smooth activations and reproducibility in deep networks
• Continual Domain-Tuning for Pretrained Language Models
• Editable Neural Networks
• Knowledge as Priors: Cross-Modal Knowledge Generalization for Datasets without Superior Knowledge
• Flood Segmentation on Sentinel-1 SAR Imagery with Semi-Supervised Learning
• Circumventing Outliers of AutoAugment with Knowledge Distillation
• Language Detection Engine for Multilingual Texting on Mobile Devices
• Towards Efficient Unconstrained Palmprint Recognition via Deep Distillation Hashing
• A Curriculum Domain Adaptation Approach to the Semantic Segmentation of Urban Scenes
• Text classification with word embedding regularization and soft similarity measure
• A Tailored Pre-Training Model for Task-Oriented Dialog Generation
• Squeezed Deep 6DoF Object Detection Using Knowledge Distillation
• Scaling Ensemble Distribution Distillation to Many Classes with Proxy Targets
• ENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translation
• Structured Pruning for Efficient ConvNets via Incremental Regularization
• Self-Enhanced GNN: Improving Graph Neural Networks Using Model Outputs
• ALP-KD: Attention-Based Layer Projection for Knowledge Distillation
• Basic Level Categorization Facilitates Visual Object Recognition
• Privacy-preserving Learning via Deep Net Pruning
• Parameterization of Cross-Token Relations with Relative Positional Encoding for Vision MLP
• GKD: Semi-supervised Graph Knowledge Distillation for Graph-Independent Inference
• Anycost GANs for Interactive Image Synthesis and Editing
• KTNet: Knowledge Transfer for Unpaired 3D Shape Completion
• Domain Adaptive Medical Image Segmentation via Adversarial Learning of Disease-Specific Spatial Patterns
• Autoencoder-Based Incremental Class Learning without Retraining on Old Data
• Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Framework
• A novel method for extracting interpretable knowledge from a spiking neural classifier with time-varying synaptic weights
• DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning
• Knowledge Distillation in Wide Neural Networks: Risk Bound, Data Efficiency and Imperfect Teacher
• Lightweight Convolutional Representations for On-Device Natural Language Processing
• Dynamic Capacity Networks
• Secost: Sequential co-supervision for large scale weakly labeled audio event detection
• AdderSR: Towards Energy Efficient Image Super-Resolution
• Trustworthy AI in the Age of Pervasive Computing and Big Data
• Smallify: Learning Network Size while Training
• The Benchmark Lottery
• Learning latent representations across multiple data domains using Lifelong VAEGAN
• Candidate Generation with Binary Codes for Large-Scale Top-N Recommendation
• Adaptive Instance Distillation for Object Detection in Autonomous Driving
• Dissecting Catastrophic Forgetting in Continual Learning by Deep Visualization
• Knowledge Extracted from Recurrent Deep Belief Network for Real Time Deterministic Control
• SST: Self-training with Self-adaptive Thresholding for Semi-supervised Learning
• Aggregate or Not? Exploring Where to Privatize in DNN Based Federated Learning Under Different Non-IID Scenes
• Advances in Medical Image Segmentation: A Comprehensive Survey with a Focus on Lumbar Spine Applications
• Preventing Catastrophic Forgetting in Continual Learning of New Natural Language Tasks
• Flow-Distilled IP Two-Stream Networks for Compressed Video Action Recognition
• Learning to Coordinate Multiple Reinforcement Learning Agents for Diverse Query Reformulation
• Latency-Constrained DNN Architecture Learning for Edge Systems using Zerorized Batch Normalization
• Learning Versatile Convolution Filters for Efficient Visual Recognition
• Decoders Matter for Semantic Segmentation: Data-Dependent Decoding Enables Flexible Feature Aggregation
• Exploring the Intersection between Neural Architecture Search and Continual Learning
• AirFace: Lightweight and Efficient Model for Face Recognition
• Low-Complexity Models for Acoustic Scene Classification Based on Receptive Field Regularization and Frequency Damping
• GAN Slimming: All-in-One GAN Compression by A Unified Optimization Framework
• SparseRT: Accelerating Unstructured Sparsity on GPUs for Deep Learning Inference
• MetaDistiller: Network Self-Boosting via Meta-Learned Top-Down Distillation
• Movie Trailer Genre Classification Using Multimodal Pretrained Features
• Deep Metric Learning Meets Deep Clustering: An Novel Unsupervised Approach for Feature Embedding
• A Roadmap to Domain Knowledge Integration in Machine Learning
• Learning Deep Nets for Gravitational Dynamics with Unknown Disturbance through Physical Knowledge Distillation: Initial Feasibility Study
• Federated Learning on Non-IID Data: A Survey
• AttendNets: Tiny Deep Image Recognition Neural Networks for the Edge via Visual Attention Condensers
• Exponential Moving Average Normalization for Self-supervised and Semi-supervised Learning
• Wireless for Machine Learning
• Generative Knowledge Transfer for Neural Language Models
• Rethinking Implicit Neural Representations for Vision Learners
• Weight Averaging: A Simple Yet Effective Method to Overcome Catastrophic Forgetting in Automatic Speech Recognition
• DINE: Domain Adaptation from Single and Multiple Black-box Predictors
• Trusted Media Challenge Dataset and User Study
• Patch-wise++ Perturbation for Adversarial Targeted Attacks
• Full-Cycle Energy Consumption Benchmark for Low-Carbon Computer Vision
• Low-rank and Sparse Soft Targets to Learn Better DNN Acoustic Models
• Joint Feature Distribution Alignment Learning for NIR-VIS and VIS-VIS Face Recognition
• A Simple and Effective Self-Supervised Contrastive Learning Framework for Aspect Detection
• Ensemble Transformer for Efficient and Accurate Ranking Tasks: an Application to Question Answering Systems
• Generation-Distillation for Efficient Natural Language Understanding in Low-Data Settings
• Test Distribution-Aware Active Learning: A Principled Approach Against Distribution Shift and Outliers
• Mosaicking to Distill: Knowledge Distillation from Out-of-Domain Data
• 1st Place Solutions of Waymo Open Dataset Challenge 2020 -- 2D Object Detection Track
• To Compress, or Not to Compress: Characterizing Deep Learning Model Compression for Embedded Inference
• Deep Spiking Neural Networks for Large Vocabulary Automatic Speech Recognition
• Prototype-based Incremental Few-Shot Semantic Segmentation
• Progressive Multi-task Anti-Noise Learning and Distilling Frameworks for Fine-grained Vehicle Recognition
• Support-Set Based Cross-Supervision for Video Grounding
• Smoothness and monotonicity constraints for neural networks using ICEnet
• Autowarp: Learning a Warping Distance from Unlabeled Time Series Using Sequence Autoencoders
• Cross-Lingual Knowledge Distillation for Answer Sentence Selection in Low-Resource Languages
• Privileged Information Dropout in Reinforcement Learning
• Sharing Attention Weights for Fast Transformer
• Leveraging Visual Supervision for Array-based Active Speaker Detection and Localization
• NaturalInversion: Data-Free Image Synthesis Improving Real-World Consistency
• Keyword localisation in untranscribed speech using visually grounded speech models
• ExpertMatcher: Automating ML Model Selection for Clients using Hidden Representations
• GraphFM: Graph Factorization Machines for Feature Interaction Modeling
• MS-KD: Multi-Organ Segmentation with Multiple Binary-Labeled Datasets
• Adaptive Confidence Smoothing for Generalized Zero-Shot Learning
• NOTE-RCNN: NOise Tolerant Ensemble RCNN for Semi-Supervised Object Detection
• Adversarial Speaker Distillation for Countermeasure Model on Automatic Speaker Verification
• SENSE: a Shared Encoder Network for Scene-flow Estimation
• Factorized Distillation: Training Holistic Person Re-identification Model by Distilling an Ensemble of Partial ReID Models
• On The State of Data In Computer Vision: Human Annotations Remain Indispensable for Developing Deep Learning Models
• On Compressing U-net Using Knowledge Distillation
• Universal Lesion Detection by Learning from Multiple Heterogeneously Labeled Datasets
• Collaboration of Experts: Achieving 80% Top-1 Accuracy on ImageNet with 100M FLOPs
• CheXseg: Combining Expert Annotations with DNN-generated Saliency Maps for X-ray Segmentation
• Grafted network for person re-identification
• Interactive Knowledge Distillation
• SGAD: Soft-Guided Adaptively-Dropped Neural Network
• Cascaded channel pruning using hierarchical self-distillation
• Convolutional Neural Network-based Topology Optimization (CNN-TO) By Estimating Sensitivity of Compliance from Material Distribution
• Improved Adversarial Robustness via Logit Regularization Methods
• Deep Learning Towards Mobile Applications
• Progressive Neural Networks for Image Classification
• A Realistic Evaluation of Semi-Supervised Learning for Fine-Grained Classification
• Sim-Real Joint Reinforcement Transfer for 3D Indoor Navigation
• The State of Knowledge Distillation for Classification
• Collaborative Teacher-Student Learning via Multiple Knowledge Transfer
• Teacher Supervises Students How to Learn From Partially Labeled Images for Facial Landmark Detection
• Facing the Hard Problems in FGVC
• Privileged Knowledge Distillation for Online Action Detection
• Task Selection Policies for Multitask Learning
• Blending LSTMs into CNNs
• PURSUhInT: In Search of Informative Hint Points Based on Layer Clustering for Knowledge Distillation
• Practical One-Shot Federated Learning for Cross-Silo Setting
• Hu-Fu: Hardware and Software Collaborative Attack Framework against Neural Networks
• Distilling EEG Representations via Capsules for Affective Computing
• Local-Selective Feature Distillation for Single Image Super-Resolution
• Teachers Do More Than Teach: Compressing Image-to-Image Models
• Accelerating Deep Learning Inference via Learned Caches
• SoK: Systematic analysis of adversarial threats against deep learning approaches for autonomous anomaly detection systems in SDN-IoT networks
• MIME: Adapting a Single Neural Network for Multi-task Inference with Memory-efficient Dynamic Pruning
• DAVE: A Unified Framework for Fast Vehicle Detection and Annotation
• Model Fusion via Optimal Transport
• Creating Something from Nothing: Unsupervised Knowledge Distillation for Cross-Modal Hashing
• NeuGuard: Lightweight Neuron-Guided Defense against Membership Inference Attacks
• Fidelity-Weighted Learning
• TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference
• Interaction-aware Factorization Machines for Recommender Systems
• EdgeMark: An Automation and Benchmarking System for Embedded Artificial Intelligence Tools
• Inspecting and Interacting with Meaningful Music Representations using VAE
• Unsupervised Hashing with Contrastive Information Bottleneck
• Teachers in concordance for pseudo-labeling of 3D sequential data
• Collaborative Distillation for Ultra-Resolution Universal Style Transfer
• Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias
• MEMO: A Deep Network for Flexible Combination of Episodic Memories
• CNN-based RGB-D Salient Object Detection: Learn, Select and Fuse
• KD-MRI: A knowledge distillation framework for image reconstruction and image restoration in MRI workflow
• Knowledge Distillation with Adaptive Asymmetric Label Sharpening for Semi-supervised Fracture Detection in Chest X-rays
• PAC Confidence Predictions for Deep Neural Network Classifiers
• General Instance Distillation for Object Detection
• DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation
• Condensation-Net: Memory-Efficient Network Architecture with Cross-Channel Pooling Layers and Virtual Feature Maps
• Understanding Continual Learning Settings with Data Distribution Drift Analysis
• Computation-Efficient Knowledge Distillation via Uncertainty-Aware Mixup
• Rethinking Network Pruning -- under the Pre-train and Fine-tune Paradigm
• ISP Distillation
• Neural Network Compression Via Sparse Optimization
• CAP-GAN: Towards Adversarial Robustness with Cycle-consistent Attentional Purification
• DADNN: Multi-Scene CTR Prediction via Domain-Aware Deep Neural Network
• Localization Distillation for Dense Object Detection
• Model Complexity of Deep Learning: A Survey
• Elsa: Energy-based learning for semi-supervised anomaly detection
• Tensor Decomposition for Compressing Recurrent Neural Network
• Towards Real-Time Neural Video Codec for Cross-Platform Application Using Calibration Information
• Revisiting Knowledge Distillation for Object Detection
• Learning Student Networks via Feature Embedding
• Analysis of the Evolution of Advanced Transformer-Based Language Models: Experiments on Opinion Mining
• Efficient Contextualized Representation: Language Model Pruning for Sequence Labeling
• VideoSSL: Semi-Supervised Learning for Video Classification
• Developing Far-Field Speaker System Via Teacher-Student Learning
• Improving the Tightness of Convex Relaxation Bounds for Training Certifiably Robust Classifiers
• Towards Off-Policy Reinforcement Learning for Ranking Policies with Human Feedback
• Learning Matching Models with Weak Supervision for Response Selection in Retrieval-based Chatbots
• Unifying and Merging Well-trained Deep Neural Networks for Inference Stage
• Learning From Less Data: Diversified Subset Selection and Active Learning in Image Classification Tasks
• Domain segmentation and adjustment for generalized zero-shot learning
• Universal Approximation with Quadratic Deep Networks
• ClusterFit: Improving Generalization of Visual Representations
• Every Node Counts: Self-Ensembling Graph Convolutional Networks for Semi-Supervised Learning
• A Scale Mixture Perspective of Multiplicative Noise in Neural Networks
• A Survey of Spatio-Temporal EEG data Analysis: from Models to Applications
• Semi-Supervising Learning, Transfer Learning, and Knowledge Distillation with SimCLR
• Learning with Interpretable Structure from Gated RNN
• VarGFaceNet: An Efficient Variable Group Convolutional Neural Network for Lightweight Face Recognition
• K-shot NAS: Learnable Weight-Sharing for NAS with K-shot Supernets
• Spatial Likelihood Voting with Self-Knowledge Distillation for Weakly Supervised Object Detection
• Real-time Memory Efficient Large-pose Face Alignment via Deep Evolutionary Network
• Essence Knowledge Distillation for Speech Recognition
• Model Compression with Two-stage Multi-teacher Knowledge Distillation for Web Question Answering System
• Knowledge Amalgamation from Heterogeneous Networks by Common Feature Learning
• Leveraging Semantic Information for Efficient Self-Supervised Emotion Recognition with Audio-Textual Distilled Models
• Recycle-and-Distill: Universal Compression Strategy for Transformer-based Speech SSL Models with Attention Map Reusing and Masking Distillation
• You Only Recognize Once: Towards Fast Video Text Spotting
• RILOD: Near Real-Time Incremental Learning for Object Detection at the Edge
• Data-Free Learning of Student Networks
• Few-Shot NLG with Pre-Trained Language Model
• Global Context for Convolutional Pose Machines
• Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoders
• Divide and Conquer: Leveraging Intermediate Feature Representations for Quantized Training of Neural Networks
• Neural Epitome Search for Architecture-Agnostic Network Compression
• Restructuring, Pruning, and Adjustment of Deep Models for Parallel Distributed Inference
• Context-Aware Multipath Networks
• Temporal Knowledge Propagation for Image-to-Video Person Re-identification
• Small Towers Make Big Differences
• Binarizing by Classification: Is soft function really necessary?
• ReabsNet: Detecting and Revising Adversarial Examples
• DELTA: A DEep learning based Language Technology plAtform
• Refactoring Neural Networks for Verification
• Importance Sampling CAMs for Weakly-Supervised Segmentation
• Auxiliary Learning for Deep Multi-task Learning
• MultiFiT: Efficient Multi-lingual Language Model Fine-tuning
• Evaluating and Boosting Uncertainty Quantification in Classification
• Students Need More Attention: BERT-based AttentionModel for Small Data with Application to AutomaticPatient Message Triage
• Techniques for Symbol Grounding with SATNet
• On the Effectiveness of Regularization Against Membership Inference Attacks
• SQUID: Deep Feature In-Painting for Unsupervised Anomaly Detection
• ColdGANs: Taming Language GANs with Cautious Sampling Strategies
• Weight Pruning via Adaptive Sparsity Loss
• Revitalizing CNN Attentions via Transformers in Self-Supervised Visual Representation Learning
• Shapley Value as Principled Metric for Structured Network Pruning
• Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation
• Cross-lingual Distillation for Text Classification
• Learning to Exploit Multiple Vision Modalities by Using Grafted Networks
• Faster ILOD: Incremental Learning for Object Detectors based on Faster RCNN
• Hide-and-Seek: A Template for Explainable AI
• Reducing the Training Time of Neural Networks by Partitioning
• Knowledge Distillation for Multilingual Unsupervised Neural Machine Translation
• Knowledge Distillation for Mobile Edge Computation Offloading
• Data-Free Knowledge Amalgamation via Group-Stack Dual-GAN
• Automatic Recall Machines: Internal Replay, Continual Learning and the Brain
• CDFI: Compression-Driven Network Design for Frame Interpolation
• A Light-weight Deep Human Activity Recognition Algorithm Using Multi-knowledge Distillation
• ESAI: Efficient Split Artificial Intelligence via Early Exiting Using Neural Architecture Search
• Bowtie Networks: Generative Modeling for Joint Few-Shot Recognition and Novel-View Synthesis
• NxMTransformer: Semi-Structured Sparsification for Natural Language Understanding via ADMM
• CoKe: Localized Contrastive Learning for Robust Keypoint Detection
• Simplified TinyBERT: Knowledge Distillation for Document Retrieval
• Attribution Preservation in Network Compression for Reliable Network Interpretation
• Multi-task Graph Convolutional Neural Network for Calcification Morphology and Distribution Analysis in Mammograms
• Structural Knowledge Distillation: Tractably Distilling Information for Structured Predictor
• RomeBERT: Robust Training of Multi-Exit BERT
• Anti-Distillation: Improving reproducibility of deep networks
• Sparsifying Neural Network Connections for Face Recognition
• Adaptive Dense-to-Sparse Paradigm for Pruning Online Recommendation System with Non-Stationary Data
• Theory-based residual neural networks: A synergy of discrete choice models and deep neural networks
• Weighted Training for Cross-Task Learning
• Making Neural Machine Reading Comprehension Faster
• Prototype Memory for Large-scale Face Representation Learning
• Representative Task Self-selection for Flexible Clustered Lifelong Learning
• Using a GAN to Generate Adversarial Examples to Facial Image Recognition
• Feature Adaptation of Pre-Trained Language Models across Languages and Domains with Robust Self-Training
• Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labels
• Towards Compact Single Image Super-Resolution via Contrastive Self-distillation
• Speech Recognition with no speech or with noisy speech
• Unifying Specialist Image Embedding into Universal Image Embedding
• Mind the Gap: Enlarging the Domain Gap in Open Set Domain Adaptation
• Efficient onboard multi-task AI architecture based on self-supervised learning
• Rearchitecting Classification Frameworks For Increased Robustness
• Fast and Accurate, Convolutional Neural Network Based Approach for Object Detection from UAV
• A Multilingual Virtual Guide for Self-Attachment Technique
• Improving End-To-End Modeling for Mispronunciation Detection with Effective Augmentation Mechanisms
• A Theory-Driven Self-Labeling Refinement Method for Contrastive Representation Learning
• Adversarial Learning with Margin-based Triplet Embedding Regularization
• Hessian-Aware Pruning and Optimal Neural Implant
• Re-distributing Biased Pseudo Labels for Semi-supervised Semantic Segmentation: A Baseline Investigation
• Backbone Can Not be Trained at Once: Rolling Back to Pre-trained Network for Person Re-Identification
• Progressive Cluster Purification for Unsupervised Feature Learning
• Exploiting Channel Similarity for Accelerating Deep Convolutional Neural Networks
• Depth-Aware Multi-Grid Deep Homography Estimation with Contextual Correlation
• Robust Reference-based Super-Resolution via C2-Matching
• Adversarial Attacks on Multivariate Time Series
• From Multimodal to Unimodal Attention in Transformers using Knowledge Distillation
• Regularizing Output Distribution of Abstractive Chinese Social Media Text Summarization for Improved Semantic Consistency
• Learning with Privileged Information for Efficient Image Super-Resolution
• ACP++: Action Co-occurrence Priors for Human-Object Interaction Detection
• UniTrans: Unifying Model Transfer and Data Transfer for Cross-Lingual Named Entity Recognition with Unlabeled Data
• Subclass Knowledge Distillation with Known Subclass Labels
• Exploring EEG and Eye Movement Fusion for Multi-Class Target RSVP-BCI
• Distilling Virtual Examples for Long-tailed Recognition
• Optimal Completion Distillation for Sequence Learning
• Learning Credible Deep Neural Networks with Rationale Regularization
• Reinforced Iterative Knowledge Distillation for Cross-Lingual Named Entity Recognition
• Data Efficient Stagewise Knowledge Distillation
• Generative Zero-shot Network Quantization
• Soft Labeling Affects Out-of-Distribution Detection of Deep Neural Networks
• Single-Modal Entropy based Active Learning for Visual Question Answering
• Knowledge Distillation for Multi-task Learning
• Ego-Exo: Transferring Visual Representations from Third-person to First-person Videos
• Distilling Knowledge from Ensembles of Acoustic Models for Joint CTC-Attention End-to-End Speech Recognition
• Biphasic Learning of GANs for High-Resolution Image-to-Image Translation
• Deep learning in bioinformatics: introduction, application, and perspective in big data era
• Teacher's pet: understanding and mitigating biases in distillation
• A Convolutional Neural Network-Based Low Complexity Filter
• Efficient Transformer-based Large Scale Language Representations using Hardware-friendly Block Structured Pruning
• 1st Place Solution to ICDAR 2021 RRC-ICTEXT End-to-end Text Spotting and Aesthetic Assessment on Integrated Circuit
• 3D-to-2D Distillation for Indoor Scene Parsing
• TTS-by-TTS: TTS-driven Data Augmentation for Fast and High-Quality Speech Synthesis
• Students are the Best Teacher: Exit-Ensemble Distillation with Multi-Exits
• Efficient Transformers in Reinforcement Learning using Actor-Learner Distillation
• CNN Acceleration by Low-rank Approximation with Quantized Factors
• Pseudo Supervised Monocular Depth Estimation with Teacher-Student Network
• How and When Adversarial Robustness Transfers in Knowledge Distillation?
• Even your Teacher Needs Guidance: Ground-Truth Targets Dampen Regularization Imposed by Self-Distillation
• Multi-head Knowledge Distillation for Model Compression
• Automated Model Compression by Jointly Applied Pruning and Quantization
• Quantile Regularization: Towards Implicit Calibration of Regression Models
• 3DQ: Compact Quantized Neural Networks for Volumetric Whole Brain Segmentation
• Robust Domain Randomised Reinforcement Learning through Peer-to-Peer Distillation
• Uncertainty-Aware Deep Calibrated Salient Object Detection
• Embedding Transfer with Label Relaxation for Improved Metric Learning
• Deep Learning and Traffic Classification: Lessons learned from a commercial-grade dataset with hundreds of encrypted and zero-day applications
• Unsupervised Tracklet Person Re-Identification
• Knowledge distillation for semi-supervised domain adaptation
• Towards Practical Lipreading with Distilled and Efficient Models
• P-KDGAN: Progressive Knowledge Distillation with GANs for One-class Novelty Detection
• Mutual Information Based Knowledge Transfer Under State-Action Dimension Mismatch
• Galileo at SemEval-2020 Task 12: Multi-lingual Learning for Offensive Language Identification using Pre-trained Language Models
• Neural Data Server: A Large-Scale Search Engine for Transfer Learning Data
• Parsing Tweets into Universal Dependencies
• Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples
• FReTAL: Generalizing Deepfake Detection using Knowledge Distillation and Representation Learning
• QuPeD: Quantized Personalization via Distillation with Applications to Federated Learning
• Resource-Constrained Federated Learning with Heterogeneous Labels and Models
• LRC-BERT: Latent-representation Contrastive Knowledge Distillation for Natural Language Understanding
• Uncertainty-Aware Multi-Shot Knowledge Distillation for Image-Based Object Re-Identification
• A Survey on Audio Synthesis and Audio-Visual Multimodal Processing
• Positive-Congruent Training: Towards Regression-Free Model Updates
• A Comparative Study of Calibration Methods for Imbalanced Class Incremental Learning
• Targeted Gradient Descent: A Novel Method for Convolutional Neural Networks Fine-tuning and Online-learning
• MimosaNet: An Unrobust Neural Network Preventing Model Stealing
• DARC: Differentiable ARchitecture Compression
• Multi-stage Progressive Compression of Conformer Transducer for On-device Speech Recognition
• Reconstructing Perceived Images from Brain Activity by Visually-guided Cognitive Representation and Adversarial Learning
• SoK: How Robust is Image Classification Deep Neural Network Watermarking? (Extended Version)
• On-the-fly Network Pruning for Object Detection
• A Stable and Effective Learning Strategy for Trainable Greedy Decoding
• CBCL-PR: A Cognitively Inspired Model for Class-Incremental Learning in Robotics
• Neural Dataset Generality
• AppealNet: An Efficient and Highly-Accurate Edge/Cloud Collaborative Architecture for DNN Inference
• Learning Architectures for Binary Networks
• NeXtVLAD: An Efficient Neural Network to Aggregate Frame-level Features for Large-scale Video Classification
• Towards a General Model of Knowledge for Facial Analysis by Multi-Source Transfer Learning
• Deep Neural Networks for Choice Analysis: Extracting Complete Economic Information for Interpretation
• Why are Sequence-to-Sequence Models So Dull? Understanding the Low-Diversity Problem of Chatbots
• Can Monolingual Pretrained Models Help Cross-Lingual Classification?
• Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Better
• Learning Dense Representations of Phrases at Scale
• CrowdTransfer: Enabling Crowd Knowledge Transfer in AIoT Community
• Relational Teacher Student Learning with Neural Label Embedding for Device Adaptation in Acoustic Scene Classification
• MarginDistillation: distillation for margin-based softmax
• Deep Ensembles on a Fixed Memory Budget: One Wide Network or Several Thinner Ones?
• Streaming egocentric action anticipation: An evaluation scheme and approach
• End-to-End Feedback Loss in Speech Chain Framework via Straight-Through Estimator
• Ensemble-Compression: A New Method for Parallel Training of Deep Neural Networks
• Interpretable Learning-to-Rank with Generalized Additive Models
• Magic Pyramid: Accelerating Inference with Early Exiting and Token Pruning
• Training Energy-Efficient Deep Spiking Neural Networks with Time-to-First-Spike Coding
• LightMBERT: A Simple Yet Effective Method for Multilingual BERT Distillation
• ABC-KD: Attention-Based-Compression Knowledge Distillation for Deep Learning-Based Noise Suppression
• LPRNet: Lightweight Deep Network by Low-rank Pointwise Residual Convolution
• Making Good on LSTMs' Unfulfilled Promise
• KDPhys: An Attention Guided 3D to 2D Knowledge Distillation for Real-time Video-Based Physiological Measurement
• Direction is what you need: Improving Word Embedding Compression in Large Language Models
• Artificial neural networks condensation: A strategy to facilitate adaption of machine learning in medical settings by reducing computational burden
• RARTS: An Efficient First-Order Relaxed Architecture Search Method
• Multi-Task Self-Training for Learning General Representations
• MAWSEO: Adversarial Wiki Search Poisoning for Illicit Online Promotion
• Edge Bias in Federated Learning and its Solution by Buffered Knowledge Distillation
• YANMTT: Yet Another Neural Machine Translation Toolkit
• Knowledge Distillation Under Ideal Joint Classifier Assumption
• Weakly Supervised 3D Object Detection from Point Clouds
• Towards Oracle Knowledge Distillation with Neural Architecture Search
• Lifelong Language Knowledge Distillation
• CL2R: Compatible Lifelong Learning Representations
• Accelerating Transformer Decoding via a Hybrid of Self-attention and Recurrent Neural Network
• Instance Cross Entropy for Deep Metric Learning
• AttriPrompter: Auto-Prompting with Attribute Semantics for Zero-shot Nuclei Detection via Visual-Language Pre-trained Models
• Informative Image Captioning with External Sources of Information
• Federated Action Recognition on Heterogeneous Embedded Devices
• Which *BERT? A Survey Organizing Contextualized Encoders
• Cross-Task Transfer for Geotagged Audiovisual Aerial Scene Recognition
• Towards Backward-Compatible Representation Learning
• Comparison of Methods Generalizing Max- and Average-Pooling
• DomBERT: Domain-oriented Language Model for Aspect-based Sentiment Analysis
• NaviSlim: Adaptive Context-Aware Navigation and Sensing via Dynamic Slimmable Networks
• Contrastive Distillation on Intermediate Representations for Language Model Compression
• BitTensor: A Peer-to-Peer Intelligence Market
• Multi-Domain Multi-Task Rehearsal for Lifelong Learning
• An Effective Deep Network for Head Pose Estimation without Keypoints
• Continual Learning for Natural Language Generation in Task-oriented Dialog Systems
• Compressing Representations for Embedded Deep Learning
• Abstraction Alignment: Comparing Model-Learned and Human-Encoded Conceptual Relationships
• Attention-Aware Noisy Label Learning for Image Classification
• Watch the Unobserved: A Simple Approach to Parallelizing Monte Carlo Tree Search
• Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition
• 3D Human Pose, Shape and Texture from Low-Resolution Images and Videos
• A Decoupled Uncertainty Model for MRI Segmentation Quality Estimation
• Binarized Neural Architecture Search for Efficient Object Recognition
• Regression Networks for Meta-Learning Few-Shot Classification
• Distilling with Performance Enhanced Students
• Unsupervised Detection of Distinctive Regions on 3D Shapes
• Manipulating Identical Filter Redundancy for Efficient Pruning on Deep and Complicated CNN
• Augmenting Neural Networks with First-order Logic
• Online Knowledge Distillation with Diverse Peers
• Learn Spelling from Teachers: Transferring Knowledge from Language Models to Sequence-to-Sequence Speech Recognition
• Manifold Regularized Dynamic Network Pruning
• PhysGAN: Generating Physical-World-Resilient Adversarial Examples for Autonomous Driving
• Knowledge Transfer via Dense Cross-Layer Mutual-Distillation
• Energy Efficiency in AI for 5G and Beyond: A DeepRx Case Study
• Periocular Embedding Learning with Consistent Knowledge Distillation from Face
• Value Function is All You Need: A Unified Learning Framework for Ride Hailing Platforms
• One-bit Supervision for Image Classification
• SeesawFaceNets: sparse and robust face verification model for mobile platform
• S2SD: Simultaneous Similarity-based Self-Distillation for Deep Metric Learning
• Learning from the Best: Rationalizing Prediction by Adversarial Information Calibration
• Simon Says: Evaluating and Mitigating Bias in Pruned Neural Networks with Knowledge Distillation
• Pre-trained Language Model for Web-scale Retrieval in Baidu Search
• Simple Distillation Baselines for Improving Small Self-supervised Models
• Lottery Jackpots Exist in Pre-trained Models
• Learning Cross-Modal Deep Representations for Multi-Modal MR Image Segmentation
• PFA: Privacy-preserving Federated Adaptation for Effective Model Personalization
• Salient Positions based Attention Network for Image Classification
• Knowledge Distillation in RNN-Attention Models for Early Prediction of Student Performance
• 3D Human Shape and Pose from a Single Low-Resolution Image with Self-Supervised Learning
• Distilling the Knowledge of BERT for Sequence-to-Sequence ASR
• Model Reuse with Reduced Kernel Mean Embedding Specification
• FQuAD2.0: French Question Answering and knowing that you know nothing
• Leveraging Just a Few Keywords for Fine-Grained Aspect Detection Through Weakly Supervised Co-Training
• Doubly Nested Network for Resource-Efficient Inference
• On Machine Learning and Structure for Mobile Robots
• Learning Invariant Representation for Continual Learning
• MutualNet: Adaptive ConvNet via Mutual Learning from Network Width and Resolution
• Robust Tracking Using Region Proposal Networks
• Jointly Optimizing Preprocessing and Inference for DNN-based Visual Analytics
• Globally Optimal Segmentation of Mutually Interacting Surfaces using Deep Learning
• Evaluating a Simple Retraining Strategy as a Defense Against Adversarial Attacks
• Detecting Dead Weights and Units in Neural Networks
• Explainable Product Search with a Dynamic Relation Embedding Model
• Cross-Task Consistency Learning Framework for Multi-Task Learning
• Triplet Distillation for Deep Face Recognition
• Detecting and Exorcising Statistical Demons from Language Models with Anti-Models of Negative Data
• Review of Video Predictive Understanding: Early Action Recognition and Future Action Prediction
• Model Extraction Attacks against Recurrent Neural Networks
• Dynamic Neural Network Channel Execution for Efficient Training
• Distilling Knowledge for Search-based Structured Prediction
• Sample-based Regularization: A Transfer Learning Strategy Toward Better Generalization
• Multilinear Compressive Learning with Prior Knowledge
• Deepening Hidden Representations from Pre-trained Language Models
• EdgeSegNet: A Compact Network for Semantic Segmentation
• Contextual Temperature for Language Modeling
• Differentiable Language Model Adversarial Attacks on Categorical Sequence Classifiers
• Continual Learning Using World Models for Pseudo-Rehearsal
• Measuring Model Complexity of Neural Networks with Curve Activation Functions
• PaCKD: Pattern-Clustered Knowledge Distillation for Compressing Memory Access Prediction Models
• Fully Learnable Group Convolution for Acceleration of Deep Neural Networks
• Improving Confidence Estimates for Unfamiliar Examples
• Computation Resource Allocation Solution in Recommender Systems
• Sparse Flows: Pruning Continuous-depth Models
• Pseudo-labeling for Scalable 3D Object Detection
• Cross-Layer Distillation with Semantic Calibration
• Double Backpropagation for Training Autoencoders against Adversarial Attack
• DeepLight: Deep Lightweight Feature Interactions for Accelerating CTR Predictions in Ad Serving
• Generalized Bayesian Posterior Expectation Distillation for Deep Neural Networks
• Why Skip If You Can Combine: A Simple Knowledge Distillation Technique for Intermediate Layers
• ShadowTutor: Distributed Partial Distillation for Mobile Video DNN Inference
• IntraQ: Learning Synthetic Images with Intra-Class Heterogeneity for Zero-Shot Network Quantization
• SAIA: Split Artificial Intelligence Architecture for Mobile Healthcare System
• Fast Video Salient Object Detection via Spatiotemporal Knowledge Distillation
• Evaluating Explanation Methods for Neural Machine Translation
• Distillation Strategies for Proximal Policy Optimization
• Continual Learning Using Multi-view Task Conditional Neural Networks
• FrugalScore: Learning Cheaper, Lighter and Faster Evaluation Metricsfor Automatic Text Generation
• Bidirectional Knowledge Reconfiguration for Lightweight Point Cloud Analysis
• Spirit Distillation: A Model Compression Method with Multi-domain Knowledge Transfer
• Trends and Advancements in Deep Neural Network Communication
• Distilling Knowledge Using Parallel Data for Far-field Speech Recognition
• Robustness Challenges in Model Distillation and Pruning for Natural Language Understanding
• A Short Study on Compressing Decoder-Based Language Models
• How Not to Give a FLOP: Combining Regularization and Pruning for Efficient Inference
• A Closer Look at Codistillation for Distributed Training
• Towards Model Agnostic Federated Learning Using Knowledge Distillation
• Bridge the Domain Gap Between Ultra-wide-field and Traditional Fundus Images via Adversarial Domain Adaptation
• Saec: Similarity-Aware Embedding Compression in Recommendation Systems
• Extract then Distill: Efficient and Effective Task-Agnostic BERT Distillation
• Neural Networks Are More Productive Teachers Than Human Raters: Active Mixup for Data-Efficient Knowledge Distillation from a Blackbox Model
• Distilling Audio-Visual Knowledge by Compositional Contrastive Learning
• Efficient Video Classification Using Fewer Frames
• Posterior Meta-Replay for Continual Learning
• Layerwise Optimization by Gradient Decomposition for Continual Learning
• Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learning
• Knowledge Distillation as Semiparametric Inference
• EBJR: Energy-Based Joint Reasoning for Adaptive Inference
• AdvPicker: Effectively Leveraging Unlabeled Data via Adversarial Discriminator for Cross-Lingual NER
• Neural Pruning via Growing Regularization
• Federated Learning with Heterogeneous Labels and Models for Mobile Activity Monitoring
• Data-Free Knowledge Distillation with Soft Targeted Transfer Set Synthesis
• Cramnet: Layer-wise Deep Neural Network Compression with Knowledge Transfer from a Teacher Network
• ProxylessKD: Direct Knowledge Distillation with Inherited Classifier for Face Recognition
• Online Knowledge Distillation via Multi-branch Diversity Enhancement
• Data Quality Matters For Adversarial Training: An Empirical Study
• Semantically-Conditioned Negative Samples for Efficient Contrastive Learning
• AlphaNet: Improved Training of Supernets with Alpha-Divergence
• Non-Autoregressive Translation by Learning Target Categorical Codes
• CEKD:Cross Ensemble Knowledge Distillation for Augmented Fine-grained Data
• Self-Supervised GAN Compression
• Weakly-Supervised Aspect-Based Sentiment Analysis via Joint Aspect-Sentiment Topic Embedding
• Connectivity Matters: Neural Network Pruning Through the Lens of Effective Sparsity
• Deep Active Learning with Augmentation-based Consistency Estimation
• An evaluation of word-level confidence estimation for end-to-end automatic speech recognition
• The Mason-Alberta Phonetic Segmenter: A forced alignment system based on deep neural networks and interpolation
• Leveraging Knowledge Distillation for Efficient Deep Reinforcement Learning in Resource-Constrained Environments
• A Simple and Robust Framework for Cross-Modality Medical Image Segmentation applied to Vision Transformers
• Efficient non-uniform quantizer for quantized neural network targeting reconfigurable hardware
• A Survey of Mobile Computing for the Visually Impaired
• PydMobileNet: Improved Version of MobileNets with Pyramid Depthwise Separable Convolution
• Generalized Ternary Connect: End-to-End Learning and Compression of Multiplication-Free Deep Neural Networks
• Knowledge distillation for optimization of quantized deep neural networks
• Self-supervised Visual Attribute Learning for Fashion Compatibility
• ExpertMatcher: Automating ML Model Selection for Users in Resource Constrained Countries
• Feature Stylization and Domain-aware Contrastive Learning for Domain Generalization
• Test-Time Adaptation for Nighttime Color-Thermal Semantic Segmentation
• AutoGAN: Robust Classifier Against Adversarial Attacks
• Neural Architecture Evolution in Deep Reinforcement Learning for Continuous Control
• PAC-Bayes Generalisation Bounds for Dynamical Systems Including Stable RNNs
• A Simple but Effective BERT Model for Dialog State Tracking on Resource-Limited Systems
• WAVER: Writing-style Agnostic Text-Video Retrieval via Distilling Vision-Language Models Through Open-Vocabulary Knowledge
• Pedestrian Detection in Low-Light Conditions: A Comprehensive Survey
• MSVQ: Self-Supervised Learning with Multiple Sample Views and Queues
• Understanding and Training Deep Diagonal Circulant Neural Networks
• Regularizing Dialogue Generation by Imitating Implicit Scenarios
• Adaptive wavelet distillation from neural networks through interpretations
• Improving Neural Topic Models with Wasserstein Knowledge Distillation
• Visual-Policy Learning through Multi-Camera View to Single-Camera View Knowledge Distillation for Robot Manipulation Tasks
• ERNIE-Tiny : A Progressive Distillation Framework for Pretrained Transformer Compression
• Transformed CNNs: recasting pre-trained convolutional layers with self-attention
• Synthesising Reinforcement Learning Policies through Set-Valued Inductive Rule Learning
• GAN-Knowledge Distillation for one-stage Object Detection
• Bridging Subword Gaps in Pretrain-Finetune Paradigm for Natural Language Generation
• Transfer Learning via Test-Time Neural Networks Aggregation
• Knowledge Distillation via Instance-level Sequence Learning
• Revisiting Knowledge Distillation: An Inheritance and Exploration Framework
• VidLanKD: Improving Language Understanding via Video-Distilled Knowledge Transfer
• Sketching and Neural Networks
• FDCNet: Feature Drift Compensation Network for Class-Incremental Weakly Supervised Object Localization
• Parameter Efficient Diverse Paraphrase Generation Using Sequence-Level Knowledge Distillation
• Textual interpretation of transient image classifications from large language models
• The Role of Teacher Calibration in Knowledge Distillation
• Co-Transport for Class-Incremental Learning
• Using Perturbed Length-aware Positional Encoding for Non-autoregressive Neural Machine Translation
• Cluster-based human-in-the-loop strategy for improving machine learning-based circulating tumor cell detection in liquid biopsy
• Content-decoupled Contrastive Learning-based Implicit Degradation Modeling for Blind Image Super-Resolution
• Weakly Supervised Lymph Nodes Segmentation Based on Partial Instance Annotations with Pre-trained Dual-branch Network and Pseudo Label Learning
• Linking Common Vulnerabilities and Exposures to the MITRE ATT&CK Framework: A Self-Distillation Approach
• LNPT: Label-free Network Pruning and Training
• DyCE: Dynamically Configurable Exiting for Deep Learning Compression and Real-time Scaling
• Flexible Variational Information Bottleneck: Achieving Diverse Compression with a Single Training
• Lifelong Intent Detection via Multi-Strategy Rebalancing
• Communication Optimization in Large Scale Federated Learning using Autoencoder Compressed Weight Updates
• Online Continual Learning For Visual Food Classification
• KMF: Knowledge-Aware Multi-Faceted Representation Learning for Zero-Shot Node Classification
• Dynamic Network Quantization for Efficient Video Inference
• Reducing the gap between streaming and non-streaming Transducer-based ASR by adaptive two-stage knowledge distillation
• Federated Learning for Open Banking
• Unsupervised Text Embedding Space Generation Using Generative Adversarial Networks for Text Synthesis
• SIGN: Spatial-information Incorporated Generative Network for Generalized Zero-shot Semantic Segmentation
• Compact representations of convolutional neural networks via weight pruning and quantization
• Edge-Cloud Collaborated Object Detection via Difficult-Case Discriminator
• Memory-Free Generative Replay For Class-Incremental Learning
• Black-Box Attacks on Sequential Recommenders via Data-Free Model Extraction
• Review learning: Real world validation of privacy preserving continual learning across medical institutions
• TriangleNet: Edge Prior Augmented Network for Semantic Segmentation through Cross-Task Consistency
• Variational Open-Domain Question Answering
• LEA-Net: Layer-wise External Attention Network for Efficient Color Anomaly Detection
• How to Select One Among All? An Extensive Empirical Study Towards the Robustness of Knowledge Distillation in Natural Language Understanding
• DS-Net++: Dynamic Weight Slicing for Efficient Inference in CNNs and Transformers
• Deep Neural Compression Via Concurrent Pruning and Self-Distillation
• Learning from learning machines: a new generation of AI technology to meet the needs of science
• Multi-label Iterated Learning for Image Classification with Label Ambiguity
• Reviewing continual learning from the perspective of human-level intelligence
• Synthetic Unknown Class Learning for Learning Unknowns
• CADA: Multi-scale Collaborative Adversarial Domain Adaptation for Unsupervised Optic Disc and Cup Segmentation
• Defining and Quantifying the Emergence of Sparse Concepts in DNNs
• Improving Binary Neural Networks through Fully Utilizing Latent Weights
• Dynamic Feature Alignment for Semi-supervised Domain Adaptation
• Dynamic Collective Intelligence Learning: Finding Efficient Sparse Model via Refined Gradients for Pruned Weights
• Fine-grained Data Distribution Alignment for Post-Training Quantization
• HALP: Hardware-Aware Latency Pruning
• When to Prune? A Policy towards Early Structural Pruning
• Response-based Distillation for Incremental Object Detection
• NeRV: Neural Representations for Videos
• Epistemic Neural Networks
• Leveraging Advantages of Interactive and Non-Interactive Models for Vector-Based Cross-Lingual Information Retrieval
• Structured Sparse R-CNN for Direct Scene Graph Generation
• Ultra-Low Power Keyword Spotting at the Edge
• Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation
• Toward Computation and Memory Efficient Neural Network Acoustic Models with Binary Weights and Activations
• An Efficient Training Approach for Very Large Scale Face Recognition
• Robustness and Diversity Seeking Data-Free Knowledge Distillation
• A Quantitative Perspective on Values of Domain Knowledge for Machine Learning
• Graph-Free Knowledge Distillation for Graph Neural Networks
• Visualizing Adapted Knowledge in Domain Transfer
• AttendSeg: A Tiny Attention Condenser Neural Network for Semantic Segmentation on the Edge
• SMRT Chatbots: Improving Non-Task-Oriented Dialog with Simulated Multiple Reference Training
• Learning Generalizable Physiological Representations from Large-scale Wearable Data
• Ultra-Lightweight Speech Separation via Group Communication
• A Selective Survey on Versatile Knowledge Distillation Paradigm for Neural Network Models
• YolactEdge: Real-time Instance Segmentation on the Edge
• Embedded Knowledge Distillation in Depth-Level Dynamic Neural Network
• Training Generative Adversarial Networks in One Stage
• Towards Open World Object Detection
• Pre-trained language models as knowledge bases for Automotive Complaint Analysis
• ADD: Augmented Disentanglement Distillation Framework for Improving Stock Trend Forecasting
• Contextual Classification Using Self-Supervised Auxiliary Models for Deep Neural Networks
• UnitedQA: A Hybrid Approach for Open Domain Question Answering
• Auto-Agent-Distiller: Towards Efficient Deep Reinforcement Learning Agents via Neural Architecture Search
• Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains
• Compressing LSTM Networks by Matrix Product Operators
• CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade
• SID: Incremental Learning for Anchor-Free Object Detection via Selective and Inter-Related Distillation
• GradNets: Dynamic Interpolation Between Neural Architectures
• On Self-Distilling Graph Neural Network
• Improved Synthetic Training for Reading Comprehension
• AACP: Model Compression by Accurate and Automatic Channel Pruning
• Universum Prescription: Regularization using Unlabeled Data
• Synthesizing Irreproducibility in Deep Networks
• ST-BERT: Cross-modal Language Model Pre-training For End-to-end Spoken Language Understanding
• Study Group Learning: Improving Retinal Vessel Segmentation Trained with Noisy Labels
• Uncertainty-aware Incremental Learning for Multi-organ Segmentation
• Beyond Self-Supervision: A Simple Yet Effective Network Distillation Alternative to Improve Backbones
• Federated Functional Gradient Boosting
• A Bayesian Evaluation Framework for Subjectively Annotated Visual Recognition Tasks
• Student Network Learning via Evolutionary Knowledge Distillation
• Representation Transfer by Optimal Transport
• Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes
• Automated Backend-Aware Post-Training Quantization
• FrostNet: Towards Quantization-Aware Network Architecture Search
• CoDiNet: Path Distribution Modeling with Consistency and Diversity for Dynamic Routing
• Pruning-then-Expanding Model for Domain Adaptation of Neural Machine Translation
• AdsGNN: Behavior-Graph Augmented Relevance Modeling in Sponsored Search
• CompOFA: Compound Once-For-All Networks for Faster Multi-Platform Deployment
• FarSee-Net: Real-Time Semantic Segmentation by Efficient Multi-scale Context Aggregation and Feature Space Super-resolution
• Analysis of Knowledge Transfer in Kernel Regime
• Structure-Level Knowledge Distillation For Multilingual Sequence Labeling
• Distilling Knowledge from Pre-trained Language Models via Text Smoothing
• Understanding the effects of artifacts on automated polyp detection and incorporating that knowledge via learning without forgetting
• On Catastrophic Interference in Atari 2600 Games
• Improving Face Recognition from Hard Samples via Distribution Distillation Loss
• ADMP: An Adversarial Double Masks Based Pruning Framework For Unsupervised Cross-Domain Compression
• Variational Metric Scaling for Metric-Based Meta-Learning
• Joint Architecture and Knowledge Distillation in CNN for Chinese Text Recognition
• Principal Component Networks: Parameter Reduction Early in Training
• Prediction Scores as a Window into Classifier Behavior
• Graph Neural Networks Including Sparse Interpretability
• Search to Distill: Pearls are Everywhere but not the Eyes
• How low can you go? Privacy-preserving people detection with an omni-directional camera
• Auto-Annotation Quality Prediction for Semi-Supervised Learning with Ensembles
• Active Subspace of Neural Networks: Structural Analysis and Universal Attacks
• Deep Semi-supervised Knowledge Distillation for Overlapping Cervical Cell Instance Segmentation
• Extreme Low Resolution Activity Recognition with Confident Spatial-Temporal Attention Transfer
• ADER: Adaptively Distilled Exemplar Replay Towards Continual Learning for Session-based Recommendation
• Bilevel Continual Learning
• Adversarial Convolutional Networks with Weak Domain-Transfer for Multi-Sequence Cardiac MR Images Segmentation
• See Clearer at Night: Towards Robust Nighttime Semantic Segmentation through Day-Night Image Conversion
• Image Captioning with Sparse Recurrent Neural Network
• Faster Person Re-Identification
• Rectified Decision Trees: Exploring the Landscape of Interpretable and Effective Machine Learning
• ICS-Assist: Intelligent Customer Inquiry Resolution Recommendation in Online Customer Service for Large E-Commerce Businesses
• Hard Pixel Mining for Depth Privileged Semantic Segmentation
• Initial Classifier Weights Replay for Memoryless Class Incremental Learning
• Cross-View Policy Learning for Street Navigation
• Revisiting the Open-Domain Question Answering Pipeline
• Weight Map Layer for Noise and Adversarial Attack Robustness
• Understanding Goal-Oriented Active Learning via Influence Functions
• Mimic and Conquer: Heterogeneous Tree Structure Distillation for Syntactic NLP
• Multi-modal Experts Network for Autonomous Driving
• Softmax Tempering for Training Neural Machine Translation Models
• NeuronBlocks: Building Your NLP DNN Models Like Playing Lego
• ATZSL: Defensive Zero-Shot Recognition in the Presence of Adversaries
• AntMan: Sparse Low-Rank Compression to Accelerate RNN inference
• Accelerating Large Scale Knowledge Distillation via Dynamic Importance Sampling
• A First Look at Deep Learning Apps on Smartphones
• Cross-modal knowledge distillation for action recognition
• Training convolutional neural networks with cheap convolutions and online distillation
• DiabDeep: Pervasive Diabetes Diagnosis based on Wearable Medical Sensors and Efficient Neural Networks
• EdgeAI: A Vision for Deep Learning in IoT Era
• Graph-Adaptive Pruning for Efficient Inference of Convolutional Neural Networks
• Repetitive Motion Estimation Network: Recover cardiac and respiratory signal from thoracic imaging
• Teacher-Student Training for Robust Tacotron-based TTS
• ReXCam: Resource-Efficient, Cross-Camera Video Analytics at Scale
• ShelfNet for Fast Semantic Segmentation
• Knowledge Distillation in Document Retrieval
• Fast Human Pose Estimation
• RePr: Improved Training of Convolutional Filters
• Unsupervised Pre-training for Natural Language Generation: A Literature Review
• A novel method for identifying the deep neural network model with the Serial Number
• Collaborative Deep Learning Across Multiple Data Centers
• Target Aware Network Adaptation for Efficient Representation Learning
• Data-Driven Compression of Convolutional Neural Networks
• Learning Compressed Transforms with Low Displacement Rank
• Low Precision Policy Distillation with Application to Low-Power, Real-time Sensation-Cognition-Action Loop with Neuromorphic Computing
• SECS: Efficient Deep Stream Processing via Class Skew Dichotomy
• Collaborative Learning for Extremely Low Bit Asymmetric Hashing
• Fine-Grained Representation Learning and Recognition by Exploiting Hierarchical Semantic Embedding
• Real-time Policy Distillation in Deep Reinforcement Learning
• Frosting Weights for Better Continual Training
• Nonparametric Bayesian Deep Networks with Local Competition
• Learning with Out-of-Distribution Data for Audio Classification
• Recent Progresses in Deep Learning based Acoustic Models (Updated)
• HENet:A Highly Efficient Convolutional Neural Networks Optimized for Accuracy, Speed and Storage
• Recent Advances in Efficient Computation of Deep Convolutional Neural Networks
• Model compression for faster structural separation of macromolecules captured by Cellular Electron Cryo-Tomography
• Deep Net Triage: Analyzing the Importance of Network Layers via Structural Compression
• Graph Distillation for Action Detection with Privileged Modalities
• Everything old is new again: A multi-view learning approach to learning using privileged information and distillation
• Visual Semantic Information Pursuit: A Survey
• Regularize, Expand and Compress: Multi-task based Lifelong Learning via NonExpansive AutoML
• WSNet: Compact and Efficient Networks Through Weight Sampling
• Guiding CTC Posterior Spike Timings for Improved Posterior Fusion and Knowledge Distillation
• Correlation Congruence for Knowledge Distillation
• An Empirical Investigation of Global and Local Normalization for Recurrent Neural Sequence Models Using a Continuous Relaxation to Beam Search
• Progressive Learning of Low-Precision Networks
• A Case for Backward Compatibility for Human-AI Teams
• HGC: Hierarchical Group Convolution for Highly Efficient Neural Network
• Unsupervised Temperature Scaling: An Unsupervised Post-Processing Calibration Method of Deep Networks
• Efficient Evaluation-Time Uncertainty Estimation by Improved Distillation
• Scalable Syntax-Aware Language Models Using Knowledge Distillation
• Self-training Converts Weak Learners to Strong Learners in Mixture Models
• Joint-DetNAS: Upgrade Your Detector with NAS, Pruning and Dynamic Distillation
• Multi-word Tokenization for Sequence Compression
• Better and Faster: Knowledge Transfer from Multiple Self-supervised Learning Tasks via Graph Distillation for Video Classification
• Learning to Embed Categorical Features without Embedding Tables for Recommendation
• Adaptive Distillation for Decentralized Learning from Heterogeneous Clients
• Efficient Action Recognition Using Confidence Distillation
• Product Image Recognition with Guidance Learning and Noisy Supervision
• Spirit Distillation: Precise Real-time Semantic Segmentation of Road Scenes with Insufficient Data
• CompNet: Neural networks growing via the compact network morphism
• Future-Guided Incremental Transformer for Simultaneous Translation
• Deep Learning in the Era of Edge Computing: Challenges and Opportunities
• A Comprehensive Approach to Unsupervised Embedding Learning based on AND Algorithm
• Performance Evaluation of Convolutional Neural Networks for Gait Recognition
• On the human-recognizability phenomenon of adversarially trained deep image classifiers
• Natural Vocabulary Emerges from Free-Form Annotations
• Unsupervised Natural Language Inference via Decoupled Multimodal Contrastive Learning
• Interventional Aspect-Based Sentiment Analysis
• Train Flat, Then Compress: Sharpness-Aware Minimization Learns More Compressible Models
• The Tensor Track VII: From Quantum Gravity to Artificial Intelligence
• Attention Mechanism with Energy-Friendly Operations
• A Unified Framework for Shot Type Classification Based on Subject Centric Lens
• Adversarial-Based Knowledge Distillation for Multi-Model Ensemble and Noisy Data Refinement
• Real-Time Decentralized knowledge Transfer at the Edge
• Large Scale Incremental Learning
• The Disruptions of 5G on Data-driven Technologies and Applications
• Voice2Mesh: Cross-Modal 3D Face Model Generation from Voices
• Balanced Binary Neural Networks with Gated Residual
• Information Theoretic Representation Distillation
• Lightweight image super-resolution with enhanced CNN
• Multi-task Self-distillation for Graph-based Semi-Supervised Learning
• Self-supervised Feature Learning via Exploiting Multi-modal Data for Retinal Disease Diagnosis
• Searching for Accurate Binary Neural Architectures
• Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression
• Towards Lifelong Self-Supervision For Unpaired Image-to-Image Translation
• Universal Model for Multi-Domain Medical Image Retrieval
• Knowledge Distillation Meets Self-Supervision
• Designing Counterfactual Generators using Deep Model Inversion
• What Deep CNNs Benefit from Global Covariance Pooling: An Optimization Perspective
• Accretionary Learning with Deep Neural Networks
• Efficient detection of adversarial images
• A Conceptual Framework for Lifelong Learning
• Distill-2MD-MTL: Data Distillation based on Multi-Dataset Multi-Domain Multi-Task Frame Work to Solve Face Related Tasksks, Multi Task Learning, Semi-Supervised Learning
• EasyTransfer -- A Simple and Scalable Deep Transfer Learning Platform for NLP Applications
• Dynamic-TinyBERT: Boost TinyBERT's Inference Efficiency by Dynamic Sequence Length
• Knowledge Integration Networks for Action Recognition
• Temporal Self-Ensembling Teacher for Semi-Supervised Object Detection
• Measuring and improving the quality of visual explanations
• Sound Event Detection by Multitask Learning of Sound Events and Scenes with Soft Scene Labels
• It Is Likely That Your Loss Should be a Likelihood
• On the Demystification of Knowledge Distillation: A Residual Network Perspective
• From Discourse to Narrative: Knowledge Projection for Event Relation Extraction
• I Have Seen Enough: A Teacher Student Network for Video Classification Using Fewer Frames
• Neural Machine Translation: A Review and Survey
• High Frequency Residual Learning for Multi-Scale Image Classification
• High-Accuracy and Low-Latency Speech Recognition with Two-Head Contextual Layer Trajectory LSTM Model
• On the Challenges of Open World Recognitionunder Shifting Visual Domains
• DCAF: A Dynamic Computation Allocation Framework for Online Serving System
• Meta-CoTGAN: A Meta Cooperative Training Paradigm for Improving Adversarial Text Generation
• Understanding Learning Dynamics of Binary Neural Networks via Information Bottleneck
• BCNet: Searching for Network Width with Bilaterally Coupled Network
• Deep Transfer Learning with Ridge Regression
• More Information Supervised Probabilistic Deep Face Embedding Learning
• Oracle Teacher: Leveraging Target Information for Better Knowledge Distillation of CTC Models
• SuperShaper: Task-Agnostic Super Pre-training of BERT Models with Variable Hidden Dimensions
• Contrast-reconstruction Representation Learning for Self-supervised Skeleton-based Action Recognition
• Paying more attention to snapshots of Iterative Pruning: Improving Model Compression via Ensemble Distillation
• Learning to Transfer Graph Embeddings for Inductive Graph based Recommendation
• Pro-KD: Progressive Distillation by Following the Footsteps of the Teacher
• Neural Audio Fingerprint for High-specific Audio Retrieval based on Contrastive Learning
• SUB-Depth: Self-distillation and Uncertainty Boosting Self-supervised Monocular Depth Estimation
• Cooperative Learning via Federated Distillation over Fading Channels
• Learning from a Lightweight Teacher for Efficient Knowledge Distillation
• Incremental Meta-Learning via Indirect Discriminant Alignment
• LiMuSE: Lightweight Multi-modal Speaker Extraction
• Adaptive Compression-based Lifelong Learning
• Regularized Evolutionary Population-Based Training
• Compression and Localization in Reinforcement Learning for ATARI Games
• Modality Distillation with Multiple Stream Networks for Action Recognition
• Solvable Model for Inheriting the Regularization through Knowledge Distillation
• BNAS v2: Learning Architectures for Binary Networks with Empirical Improvements
• Pretraining Image Encoders without Reconstruction via Feature Prediction Loss
• Learning from Noisy Labels with Noise Modeling Network
• On Vocabulary Reliance in Scene Text Recognition
• Efficient Crowd Counting via Structured Knowledge Transfer
• Entropy-Based Uncertainty Calibration for Generalized Zero-Shot Learning
• Spatio-Temporal Channel Correlation Networks for Action Classification
• Differentiable Neural Architecture Learning for Efficient Neural Network Design
• Formal Methods with a Touch of Magic
• Channel Pruning via Optimal Thresholding
• Filter Pruning using Hierarchical Group Sparse Regularization for Deep Convolutional Neural Networks
• DECORE: Deep Compression with Reinforcement Learning
• Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task
• Developing Multi-Task Recommendations with Long-Term Rewards via Policy Distilled Reinforcement Learning
• Representation Consolidation for Training Expert Students
• More Industry-friendly: Federated Learning with High Efficient Design
• Follow Your Path: a Progressive Method for Knowledge Distillation
• A New Angle on L2 Regularization
• Efficient Encrypted Inference on Ensembles of Decision Trees
• Efficient Facial Feature Learning with Wide Ensemble-based Convolutional Neural Networks
• Lightweight 3D Human Pose Estimation Network Training Using Teacher-Student Learning
• Churn Reduction via Distillation
• Selective Knowledge Distillation for Neural Machine Translation
• Bridging In- and Out-of-distribution Samples for Their Better Discriminability
• A Robust Classification-autoencoder to Defend Outliers and Adversaries
• Scenario-aware and Mutual-based approach for Multi-scenario Recommendation in E-Commerce
• Recurrent Stacking of Layers for Compact Neural Machine Translation Models
• Recent Advances in Convolutional Neural Network Acceleration
• Curriculum Learning Strategies for IR: An Empirical Study on Conversation Response Ranking
• Unsupervised Out-of-Domain Detection via Pre-trained Transformers
• On Focal Loss for Class-Posterior Probability Estimation: A Theoretical Perspective
• EagleEye: Attack-Agnostic Defense against Adversarial Inputs (Technical Report)
• GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
• Incremental Learning in Person Re-Identification
• Dual-Stream Spectral Decoupling Distillation for Remote Sensing Object Detection
• Co-advise: Cross Inductive Bias Distillation
• Improving image generative models with human interactions
• MITAS: A Compressed Time-Domain Audio Separation Network with Parameter Sharing
• Triple-cooperative Video Shadow Detection
• Scaling Video Analytics Systems to Large Camera Deployments
• Improving Task-Agnostic BERT Distillation with Layer Mapping Search
• StackNet: Stacking Parameters for Continual learning
• SMYRF: Efficient Attention using Asymmetric Clustering
• Path-based knowledge reasoning with textual semantic information for medical knowledge graph completion
• Text is Text, No Matter What: Unifying Text Recognition using Knowledge Distillation
• BERT for Large-scale Video Segment Classification with Test-time Augmentation
• Preliminary Steps Towards Federated Sentiment Classification
• Cross-Modal Knowledge Distillation Method for Automatic Cued Speech Recognition
• I'm Sorry for Your Loss: Spectrally-Based Audio Distances Are Bad at Pitch
• Transparent Classification with Multilayer Logical Perceptrons and Random Binarization
• Training Sparse Neural Networks using Compressed Sensing
• NewsBERT: Distilling Pre-trained Language Model for Intelligent News Application
• Real-Time Cell Sorting with Scalable In Situ FPGA-Accelerated Deep Learning
• Deep Consensus Learning
• Matching Guided Distillation
• Robustly Optimized and Distilled Training for Natural Language Understanding
• Distilling the Knowledge from Conditional Normalizing Flows
• Iterative Distillation for Better Uncertainty Estimates in Multitask Emotion Recognition
• Black-Box Dissector: Towards Erasing-based Hard-Label Model Stealing Attack
• Visually Imperceptible Adversarial Patch Attacks on Digital Images
• Iterative Window Mean Filter: Thwarting Diffusion-based Adversarial Purification
• TSCM: A Teacher-Student Model for Vision Place Recognition Using Cross-Metric Knowledge Distillation
• Pay attention! - Robustifying a Deep Visuomotor Policy through Task-Focused Attention
• GroupBERT: Enhanced Transformer Architecture with Efficient Grouped Structures
• DarkGAN: Exploiting Knowledge Distillation for Comprehensible Audio Synthesis with GANs
• Multilevel Knowledge Transfer for Cross-Domain Object Detection
• Dropout as a Regularizer of Interaction Effects
• Prioritized Architecture Sampling with Monto-Carlo Tree Search
• Go From the General to the Particular: Multi-Domain Translation with Domain Transformation Networks
• On the Exploitability of Audio Machine Learning Pipelines to Surreptitious Adversarial Examples
• Teacher-Explorer-Student Learning: A Novel Learning Method for Open Set Recognition
• RankDVQA-mini: Knowledge Distillation-Driven Deep Video Quality Assessment
• Universal Adder Neural Networks
• Three Dimensional Convolutional Neural Network Pruning with Regularization-Based Method
• Active multi-fidelity Bayesian online changepoint detection
• Gradient Boosted Decision Tree Neural Network
• On Modelling Label Uncertainty in Deep Neural Networks: Automatic Estimation of Intra-observer Variability in 2D Echocardiography Quality Assessment
• Tatum-Level Drum Transcription Based on a Convolutional Recurrent Neural Network with Language Model-Based Regularized Training
• Graph-Based Continual Learning
• Deep Neural Network Approximation using Tensor Sketching
• Sequence-Level Training for Non-Autoregressive Neural Machine Translation
• ReCU: Reviving the Dead Weights in Binary Neural Networks
• Learn Faster and Forget Slower via Fast and Stable Task Adaptation
• KD-Lib: A PyTorch library for Knowledge Distillation, Pruning and Quantization
• Recurrent Convolution for Compact and Cost-Adjustable Neural Networks: An Empirical Study
• One-bit Supervision for Image Classification: Problem, Solution, and Beyond
• Prototype-based Personalized Pruning
• ATCN: Resource-Efficient Processing of Time Series on Edge
• Sample Efficient Learning of Image-Based Diagnostic Classifiers Using Probabilistic Labels
• Distilling Pixel-Wise Feature Similarities for Semantic Segmentation
• Distilling Black-Box Travel Mode Choice Model for Behavioral Interpretation
• Sequence-Level Knowledge Distillation for Model Compression of Attention-based Sequence-to-Sequence Speech Recognition
• Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Time
• Interpreting and Disentangling Feature Components of Various Complexity from DNNs
• Adviser Networks: Learning What Question to Ask for Human-In-The-Loop Viewpoint Estimation
• FP-NAS: Fast Probabilistic Neural Architecture Search
• Convolutional Tables Ensemble: classification in microseconds
• Doubly Sparse: Sparse Mixture of Sparse Experts for Efficient Softmax Inference
• MOD: A Deep Mixture Model with Online Knowledge Distillation for Large Scale Video Temporal Concept Localization
• Fractional Transfer Learning for Deep Model-Based Reinforcement Learning
• Fast Adjustable Threshold For Uniform Neural Network Quantization (Winning solution of LPIRC-II)
• URNet : User-Resizable Residual Networks with Conditional Gating Module
• Impression-Aware Recommender Systems
• Neural Program Synthesis By Self-Learning
• Text Analytics for Resilience-Enabled Extreme Events Reconnaissance
• Multi-view Contrastive Learning for Online Knowledge Distillation
• A Fast Knowledge Distillation Framework for Visual Recognition
• Exploiting Large-scale Teacher-Student Training for On-device Acoustic Models
• Dual policy as self-model for planning
• Fixing the Teacher-Student Knowledge Discrepancy in Distillation
• The Unreasonable Effectiveness of Large Language-Vision Models for Source-free Video Domain Adaptation
• SM-SGE: A Self-Supervised Multi-Scale Skeleton Graph Encoding Framework for Person Re-Identification
• Learning to Remember from a Multi-Task Teacher
• Going Beyond Classification Accuracy Metrics in Model Compression
• Multi-Step Adversarial Perturbations on Recommender Systems Embeddings
• Self-Supervised Transformer Architecture for Change Detection in Radio Access Networks
• Training Quantized Neural Networks with a Full-precision Auxiliary Module
• Multi-view Subword Regularization
• TinyGAN: Distilling BigGAN for Conditional Image Generation
• Content-Aware GAN Compression
• Knowledge Distillation for Small-footprint Highway Networks
• Adversarial Robustness for Unsupervised Domain Adaptation
• A General Class of Transfer Learning Regression without Implementation Cost
• Prior knowledge distillation based on financial time series
• Unveiling the Tapestry: the Interplay of Generalization and Forgetting in Continual Learning
• Embracing the Dark Knowledge: Domain Generalization Using Regularized Knowledge Distillation
• ThumbNet: One Thumbnail Image Contains All You Need for Recognition
• BasisConv: A method for compressed representation and learning in CNNs
• On the Orthogonality of Knowledge Distillation with Other Techniques: From an Ensemble Perspective
• Extending Label Smoothing Regularization with Self-Knowledge Distillation
• Edge: Enriching Knowledge Graph Embeddings with External Text
• KSM: Fast Multiple Task Adaption via Kernel-wise Soft Mask Learning
• Learning Light-Weight Translation Models from Deep Transformer
• Semi-Anchored Detector for One-Stage Object Detection
• Sentence Embeddings by Ensemble Distillation
• Let's Enhance: A Deep Learning Approach to Extreme Deblurring of Text Images
• Efficient Object Embedding for Spliced Image Retrieval
• Incremental Concept Learning via Online Generative Memory Recall
• End-to-End Automatic Speech Recognition with Deep Mutual Learning
• RECALL: Rehearsal-free Continual Learning for Object Classification
• MSSRNet: Manipulating Sequential Style Representation for Unsupervised Text Style Transfer
• Leveraging Structured Pruning of Convolutional Neural Networks
• Ensemble Manifold Segmentation for Model Distillation and Semi-supervised Learning
• Point Adversarial Self Mining: A Simple Method for Facial Expression Recognition
• Dirichlet Pruning for Neural Network Compression
• $k$-Neighbor Based Curriculum Sampling for Sequence Prediction
• Momentum Contrast Speaker Representation Learning
• Towards Inference Delivery Networks: Distributing Machine Learning with Optimality Guarantees
• Training Flexible Depth Model by Multi-Task Learning for Neural Machine Translation
• An Approximation Algorithm for Optimal Subarchitecture Extraction
• Towards Accurate Quantization and Pruning via Data-free Knowledge Transfer
• AutoADR: Automatic Model Design for Ad Relevance
• IC Networks: Remodeling the Basic Unit for Convolutional Neural Networks
• Adversarial Self-Supervised Data-Free Distillation for Text Classification
• Improving QA Generalization by Concurrent Modeling of Multiple Biases
• Q-Rater: Non-Convex Optimization for Post-Training Uniform Quantization
• Improved Customer Transaction Classification using Semi-Supervised Knowledge Distillation
• A Fully Tensorized Recurrent Neural Network
• Knowledge Distillation for Improved Accuracy in Spoken Question Answering
• Two-stage Textual Knowledge Distillation for End-to-End Spoken Language Understanding
• BasisNet: Two-stage Model Synthesis for Efficient Inference
• Designing Interpretable Approximations to Deep Reinforcement Learning
• Can the state of relevant neurons in a deep neural networks serve as indicators for detecting adversarial attacks?
• Benchmarking Approximate Inference Methods for Neural Structured Prediction
• TIE: A Framework for Embedding-based Incremental Temporal Knowledge Graph Completion
• Enhancing Data-Free Adversarial Distillation with Activation Regularization and Virtual Interpolation
• What Can You Learn from Your Muscles? Learning Visual Representation from Human Interactions
• A Multiclass Boosting Framework for Achieving Fast and Provable Adversarial Robustness
• A Procedural World Generation Framework for Systematic Evaluation of Continual Learning
• You Never Cluster Alone
• Denoising Large-Scale Image Captioning from Alt-text Data using Content Selection Models
• Cocktail: Learn a Better Neural Network Controller from Multiple Experts via Adaptive Mixing and Robust Distillation
• CubiCasa5K: A Dataset and an Improved Multi-Task Model for Floorplan Image Analysis
• A Deep Value-network Based Approach for Multi-Driver Order Dispatching
• Rethinking the Knowledge Distillation From the Perspective of Model Calibration
• AKE-GNN: Effective Graph Learning with Adaptive Knowledge Exchange
• Dialogue Distillation: Open-Domain Dialogue Augmentation Using Unpaired Data
• Revisiting Catastrophic Forgetting in Class Incremental Learning
• Efficient Inference via Universal LSH Kernel
• Knowledge Distillation-aided End-to-End Learning for Linear Precoding in Multiuser MIMO Downlink Systems with Finite-Rate Feedback
• Similarity Transfer for Knowledge Distillation
• Partition and Code: learning how to compress graphs
• A novel multimodal fusion network based on a joint coding model for lane line segmentation
• Dynamic Slimmable Network
• The Difficulty of Passive Learning in Deep Reinforcement Learning
• Knowledge Distillation For Recurrent Neural Network Language Modeling With Trust Regularization
• Rapid Elastic Architecture Search under Specialized Classes and Resource Constraints
• A New Training Framework for Deep Neural Network
• Real-Time Uncertainty Estimation in Computer Vision via Uncertainty-Aware Distribution Distillation
• Resolution Switchable Networks for Runtime Efficient Image Recognition
• Adversarially-learned Inference via an Ensemble of Discrete Undirected Graphical Models
• Progressive Multi-stage Feature Mix for Person Re-Identification
• Instance-Dependent Partial Label Learning
• Deep Ensemble Collaborative Learning by using Knowledge-transfer Graph for Fine-grained Object Classification
• Adjoined Networks: A Training Paradigm with Applications to Network Compression
• Neural Parameter Allocation Search
• Complementary Relation Contrastive Distillation
• The Heterogeneity Hypothesis: Finding Layer-Wise Differentiated Network Architectures
• Federated Few-Shot Learning with Adversarial Learning
• Unsupervised Domain Expansion for Visual Categorization
• Adaptive Distillation: Aggregating Knowledge from Multiple Paths for Efficient Distillation
• Streaming Transformer ASR with Blockwise Synchronous Beam Search
• Deep Learning: Our Miraculous Year 1990-1991
• Matrix Smoothing: A Regularization for DNN with Transition Matrix under Noisy Labels
• DJEnsemble: On the Selection of a Disjoint Ensemble of Deep Learning Black-Box Spatio-Temporal Models
• SIPA: A Simple Framework for Efficient Networks
• "BNN - BN = ?": Training Binary Neural Networks without Batch Normalization
• Blind Adversarial Pruning: Balance Accuracy, Efficiency and Robustness
• Prune Your Model Before Distill It
• Data-Efficient Language-Supervised Zero-Shot Learning with Self-Distillation
• Dual-View Distilled BERT for Sentence Embedding
• Prune2Edge: A Multi-Phase Pruning Pipelines to Deep Ensemble Learning in IIoT
• Compact CNN Structure Learning by Knowledge Distillation
• Cost-effective Deployment of BERT Models in Serverless Environment
• Selective Replay Enhances Learning in Online Continual Analogical Reasoning
• Task-adaptive Pre-training and Self-training are Complementary for Natural Language Understanding
• Brittle Features May Help Anomaly Detection
• Towards Non-task-specific Distillation of BERT via Sentence Representation Approximation
• Transfer Learning in Automated Gamma Spectral Identification
• DP-Net: Dynamic Programming Guided Deep Neural Network Compression
• Decoupled Gradient Harmonized Detector for Partial Annotation: Application to Signet Ring Cell Detection
• Dynamic Slimmable Denoising Network
• DEPARA: Deep Attribution Graph for Deep Knowledge Transferability
• Iterative Teaching by Label Synthesis
• One Model to Recognize Them All: Marginal Distillation from NER Models with Different Tag Sets
• Sparse MoEs meet Efficient Ensembles
• A Strong Baseline for Semi-Supervised Incremental Few-Shot Learning
• General Purpose Text Embeddings from Pre-trained Language Models for Scalable Inference
• Light-Field Microscopy for optical imaging of neuronal activity: when model-based methods meet data-driven approaches
• Probabilistic Object Classification using CNN ML-MAP layers
• Generative Feature Replay with Orthogonal Weight Modification for Continual Learning
• Introducing Pose Consistency and Warp-Alignment for Self-Supervised 6D Object Pose Estimation in Color Images
• False Negative Distillation and Contrastive Learning for Personalized Outfit Recommendation
• Robust and Information-theoretically Safe Bias Classifier against Adversarial Attacks
• A Study of Non-autoregressive Model for Sequence Generation
• Inclusive Design: Accessibility Settings for People with Cognitive Disabilities
• Homophone-based Label Smoothing in End-to-End Automatic Speech Recognition
• Arch-Net: Model Distillation for Architecture Agnostic Model Deployment
• Structured Sparsification with Joint Optimization of Group Convolution and Channel Shuffle
• PANet: Perspective-Aware Network with Dynamic Receptive Fields and Self-Distilling Supervision for Crowd Counting
• Defending Adversarial Attacks via Semantic Feature Manipulation
• Learning Multi-granular Quantized Embeddings for Large-Vocab Categorical Features in Recommender Systems
• Self-Distilled Self-Supervised Representation Learning
• MTSS: Learn from Multiple Domain Teachers and Become a Multi-domain Dialogue Expert
• Unsupervised Domain Expansion from Multiple Sources
• Mining the Weights Knowledge for Optimizing Neural Network Structures
• Fully Learnable Front-End for Multi-Channel Acoustic Modeling using Semi-Supervised Learning
• Creating Lightweight Object Detectors with Model Compression for Deployment on Edge Devices
• Enhance Long Text Understanding via Distilled Gist Detector from Abstractive Summarization
• Tensor train decompositions on recurrent networks
• Convolutional neural networks compression with low rank and sparse tensor decompositions
• CGCL: Collaborative Graph Contrastive Learning without Handcrafted Graph Data Augmentations
• Measure Twice, Cut Once: Quantifying Bias and Fairness in Deep Neural Networks
• Semi-Streaming Architecture: A New Design Paradigm for CNN Implementation on FPGAs
• Certifying Strategyproof Auction Networks
• Domain Adaptation Regularization for Spectral Pruning
• Efficient Micro-Structured Weight Unification and Pruning for Neural Network Compression
• Rapid Structural Pruning of Neural Networks with Set-based Task-Adaptive Meta-Pruning
• Model Compression for Resource-Constrained Mobile Robots
• Globally Optimal Surface Segmentation using Deep Learning with Learnable Smoothness Priors
• Maximum Entropy Regularization and Chinese Text Recognition
• Patient-Specific Domain Adaptation for Fast Optical Flow Based on Teacher-Student Knowledge Transfer
• (DE)^2 CO: Deep Depth Colorization
• Adversarial Distillation for Ordered Top-k Attacks
• Semi-Online Knowledge Distillation
• One to Transfer All: A Universal Transfer Framework for Vision Foundation Model with Few Data
• Non-Determinism in Neural Networks for Adversarial Robustness
• Defocus Blur Detection via Depth Distillation
• Abductive Reasoning as Self-Supervision for Common Sense Question Answering
• ParaDiS: Parallelly Distributable Slimmable Neural Networks
• Scale Calibrated Training: Improving Generalization of Deep Networks via Scale-Specific Normalization
• Fully Dynamic Inference with Deep Neural Networks
• Amalgamating Filtered Knowledge: Learning Task-customized Student from Multi-task Teachers
• Tiny-NewsRec: Effective and Efficient PLM-based News Recommendation
• Light Field Saliency Detection with Dual Local Graph Learning andReciprocative Guidance
• Ultra-light deep MIR by trimming lottery tickets
• Knowledge Transfer Graph for Deep Collaborative Learning
• Semi-tensor Product-based TensorDecomposition for Neural Network Compression
• Differentiable Feature Aggregation Search for Knowledge Distillation
• Prime-Aware Adaptive Distillation
• Don't ignore Dropout in Fully Convolutional Networks
• Dynamic Knowledge Distillation for Black-box Hypothesis Transfer Learning
• The Pupil Has Become the Master: Teacher-Student Model-Based Word Embedding Distillation with Ensemble Learning
• Cross-Lingual Language Model Meta-Pretraining
• Driving among Flatmobiles: Bird-Eye-View occupancy grids from a monocular camera for holistic trajectory planning
• Full-Stack Filters to Build Minimum Viable CNNs
• Knowledge Distillation and Data Selection for Semi-Supervised Learning in CTC Acoustic Models
• The Partial Response Network: a neural network nomogram
• MobileFAN: Transferring Deep Hidden Representation for Face Alignment
• A Hierarchical User Intention-Habit Extract Network for Credit Loan Overdue Risk Detection
• Energy Consumption Analysis of pruned Semantic Segmentation Networks on an Embedded GPU
• Weakly Supervised Dense Video Captioning via Jointly Usage of Knowledge Distillation and Cross-modal Matching
• Deep Learning Acceleration Techniques for Real Time Mobile Vision Applications
• Privileged Features Distillation at Taobao Recommendations
• LioNets: Local Interpretation of Neural Networks through Penultimate Layer Decoding
• Bias-Awareness for Zero-Shot Learning the Seen and Unseen
• New Directions in Distributed Deep Learning: Bringing the Network at Forefront of IoT Design
• Exploiting Features and Logits in Heterogeneous Federated Learning
• Webly Supervised Image Classification with Self-Contained Confidence
• Dynamic Resolution Network
• LabelEnc: A New Intermediate Supervision Method for Object Detection
• Private Deep Learning with Teacher Ensembles
• Cross-domain Cross-architecture Black-box Attacks on Fine-tuned Models with Transferred Evolutionary Strategies
• Neural Crossbreed: Neural Based Image Metamorphosis
• Mixture-of-Partitions: Infusing Large Biomedical Knowledge Graphs into BERT
• Dynamic Routing Networks
• Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation
• Artificial Intelligence for 5G Wireless Systems: Opportunities, Challenges, and Future Research Directions
• Recursive Sketches for Modular Deep Learning
• Introspective Learning by Distilling Knowledge from Online Self-explanation
• MONCAE: Multi-Objective Neuroevolution of Convolutional Autoencoders
• Feature Distillation With Guided Adversarial Contrastive Learning
• Open-set Short Utterance Forensic Speaker Verification using Teacher-Student Network with Explicit Inductive Bias
• Building Compact and Robust Deep Neural Networks with Toeplitz Matrices
• Energy Aligning for Biased Models
• Efficient Person Search: An Anchor-Free Approach
• Pruning with Compensation: Efficient Channel Pruning for Deep Convolutional Neural Networks
• Mobiprox: Supporting Dynamic Approximate Computing on Mobiles
• Understanding the Logit Distributions of Adversarially-Trained Deep Neural Networks
• A scalable convolutional neural network for task-specified scenarios via knowledge distillation
• Learning to estimate label uncertainty for automatic radiology report parsing
• Certainty Driven Consistency Loss on Multi-Teacher Networks for Semi-Supervised Learning
• Heterogeneous Domain Adaptation with Positive and Unlabeled Data
• Learning Effective and Efficient Embedding via an Adaptively-Masked Twins-based Layer
• Convolutional Prototype Learning for Zero-Shot Recognition
• Efficient and Robust Machine Learning for Real-World Systems
• EEG aided boosting of single-lead ECG based sleep staging with Deep Knowledge Distillation
• Neural Rejuvenation: Improving Deep Network Training by Enhancing Computational Resource Utilization
• A Light-weight contextual spelling correction model for customizing transducer-based speech recognition systems
• Interpretable Disentanglement of Neural Networks by Extracting Class-Specific Subnetwork
• From Consensus to Disagreement: Multi-Teacher Distillation for Semi-Supervised Relation Extraction
• DIODE: Dilatable Incremental Object Detection
• PyEuroVoc: A Tool for Multilingual Legal Document Classification with EuroVoc Descriptors
• torchdistill Meets Hugging Face Libraries for Reproducible, Coding-Free Deep Learning Studies: A Case Study on NLP
• An EM Framework for Online Incremental Learning of Semantic Segmentation
• Improving Span-based Question Answering Systems with Coarsely Labeled Data
• FedRAD: Federated Robust Adaptive Distillation
• Mixture of Expert/Imitator Networks: Scalable Semi-supervised Learning Framework
• Composite Binary Decomposition Networks
• Distilling Transformers for Neural Cross-Domain Search
• An Empirical Study of Generation Order for Machine Translation
• To Beam Or Not To Beam: That is a Question of Cooperation for Language GANs
• Reducing audio membership inference attack accuracy to chance: 4 defenses
• When Video Classification Meets Incremental Classes
• Learning-based Framework for Sensor Fault-Tolerant Building HVAC Control with Model-assisted Learning
• Knowledge Representing: Efficient, Sparse Representation of Prior Knowledge for Knowledge Distillation
• Recognizing Facial Expressions in the Wild using Multi-Architectural Representations based Ensemble Learning with Distillation
• Multiple Style-Transfer in Real-Time
• A Miniaturized Semantic Segmentation Method for Remote Sensing Image
• MergeDistill: Merging Pre-trained Language Models using Distillation
• Improving Forward Compatibility in Class Incremental Learning by Increasing Representation Rank and Feature Richness
• Smell Pittsburgh: Community-Empowered Mobile Smell Reporting System
• UniEmoX: Cross-modal Semantic-Guided Large-Scale Pretraining for Universal Scene Emotion Perception
• Why Blocking Targeted Adversarial Perturbations Impairs the Ability to Learn
• FLASH: Fast Neural Architecture Search with Hardware Optimization
• Diversifying Dialog Generation via Adaptive Label Smoothing
• An Exploration of Mimic Architectures for Residual Network Based Spectral Mapping
• Diversifying Inference Path Selection: Moving-Mobile-Network for Landmark Recognition
• WaLDORf: Wasteless Language-model Distillation On Reading-comprehension
• Search-Optimized Quantization in Biomedical Ontology Alignment
• FedPromo: Federated Lightweight Proxy Models at the Edge Bring New Domains to Foundation Models
• Advancing Multi-Accented LSTM-CTC Speech Recognition using a Domain Specific Student-Teacher Learning Paradigm
• RDPD: Rich Data Helps Poor Data via Imitation
• Towards Sustainable Artificial Intelligence: A Comprehensive Review and Comparative Analysis of Deep Learning Models' Carbon Footprint
• Review-Driven Multi-Label Music Style Classification by Exploiting Style Correlations
• Text-to-image Synthesis via Symmetrical Distillation Networks
• Training Classifiers that are Universally Robust to All Label Noise Levels
• RAM-NAS: Resource-aware Multiobjective Neural Architecture Search Method for Robot Vision Tasks
• Action Anticipation By Predicting Future Dynamic Images
• Robust Student Network Learning
• Distilling Stereo Networks for Performant and Efficient Leaner Networks
• Extracting Weighted Automata for Approximate Minimization in Language Modelling
• DeGAN : Data-Enriching GAN for Retrieving Representative Samples from a Trained Classifier
• Domain Adaptation via Teacher-Student Learning for End-to-End Speech Recognition
• Similarity Based Label Smoothing For Dialogue Generation
• MultiTASC++: A Continuously Adaptive Scheduler for Edge-Based Multi-Device Cascade Inference
• Model Fusion via Neuron Transplantation
• Pushing the boundaries of parallel Deep Learning -- A practical approach
• Learning ULMFiT and Self-Distillation with Calibration for Medical Dialogue System
• Training for temporal sparsity in deep neural networks, application in video processing
• Periodic Intra-Ensemble Knowledge Distillation for Reinforcement Learning
• EIGEN: Ecologically-Inspired GENetic Approach for Neural Network Structure Searching from Scratch
• FastSeq: Make Sequence Generation Faster
• SQWA: Stochastic Quantized Weight Averaging for Improving the Generalization Capability of Low-Precision Deep Neural Networks
• Deep Pepper: Expert Iteration based Chess agent in the Reinforcement Learning Setting
• Aligning the Pretraining and Finetuning Objectives of Language Models
• Transfer Heterogeneous Knowledge Among Peer-to-Peer Teammates: A Model Distillation Approach
• Widening and Squeezing: Towards Accurate and Efficient QNNs
• Balancing Cost and Benefit with Tied-Multi Transformers
• Regularization-based Continual Learning for Fault Prediction in Lithium-Ion Batteries
• Quantization Mimic: Towards Very Tiny CNN for Object Detection
• An Efficient Method of Training Small Models for Regression Problems with Knowledge Distillation
• Precise Box Score: Extract More Information from Datasets to Improve the Performance of Face Detection
• Adaptive Beam Search to Enhance On-device Abstractive Summarization
• Categorical Relation-Preserving Contrastive Knowledge Distillation for Medical Image Classification
• Energy-efficient and Robust Cumulative Training with Net2Net Transformation
• Training a Binary Weight Object Detector by Knowledge Transfer for Autonomous Driving
• On The Distribution of Penultimate Activations of Classification Networks
• Sampling-free Uncertainty Estimation in Gated Recurrent Units with Exponential Families
• Substitute Teacher Networks: Learning with Almost No Supervision
• A Main/Subsidiary Network Framework for Simplifying Binary Neural Network
• KnowSR: Knowledge Sharing among Homogeneous Agents in Multi-agent Reinforcement Learning
• Scaling shared model governance via model splitting
• Studying the Plasticity in Deep Convolutional Neural Networks using Random Pruning
• Pool of Experts: Realtime Querying Specialized Knowledge in Massive Neural Networks
• Continual Learning Augmented Investment Decisions
• Progressive Label Distillation: Learning Input-Efficient Deep Neural Networks
• Dynamic Multi-path Neural Network
• When Side-Channel Attacks Break the Black-Box Property of Embedded Artificial Intelligence
• GridDehazeNet+: An Enhanced Multi-Scale Network with Intra-Task Knowledge Transfer for Single Image Dehazing
• Multimodal Knowledge Expansion
• DiffPrune: Neural Network Pruning with Deterministic Approximate Binary Gates and $L_0$ Regularization
• Incremental Learning via Rate Reduction
• Improved Techniques for Quantizing Deep Networks with Adaptive Bit-Widths
• Improving Sequential Recommendation Consistency with Self-Supervised Imitation
• An In-depth Analysis of Passage-Level Label Transfer for Contextual Document Ranking
• Pre-trained Language Model based Ranking in Baidu Search
• Digging Deeper into CRNN Model in Chinese Text Images Recognition
• Irregularly Tabulated MLP for Fast Point Feature Embedding
• Data-free mixed-precision quantization using novel sensitivity metric
• Unsupervised Continual Learning Via Pseudo Labels
• Dual Discriminator Adversarial Distillation for Data-free Model Compression
• Learning by Distillation: A Self-Supervised Learning Framework for Optical Flow Estimation
• Teacher-Student Asynchronous Learning with Multi-Source Consistency for Facial Landmark Detection
• Towards Recognizing New Semantic Concepts in New Visual Domains
• Spending Your Winning Lottery Better After Drawing It
• Human Activity Recognition for Edge Devices
• Learning to Augment for Data-Scarce Domain BERT Knowledge Distillation
• Augmentation Inside the Network
• What Synthesis is Missing: Depth Adaptation Integrated with Weak Supervision for Indoor Scene Parsing
• Knowledge Transfer Based Fine-grained Visual Classification
• Improving Route Choice Models by Incorporating Contextual Factors via Knowledge Distillation
• Data-free Knowledge Distillation for Segmentation using Data-Enriching GAN
• Learning Interpretable End-to-End Vision-Based Motion Planning for Autonomous Driving with Optical Flow Distillation
• Multi-level Knowledge Distillation via Knowledge Alignment and Correlation
• EQ-Net: A Unified Deep Learning Framework for Log-Likelihood Ratio Estimation and Quantization
• Compositionality Through Language Transmission, using Artificial Neural Networks
• Label Augmentation via Time-based Knowledge Distillation for Financial Anomaly Detection
• Learning Unified Embedding for Apparel Recognition
• Rethinking Ensemble-Distillation for Semantic Segmentation Based Unsupervised Domain Adaptation
• Learning Robust Variational Information Bottleneck with Reference
• Person Search Challenges and Solutions: A Survey
• Unsupervised Noisy Tracklet Person Re-identification
• A Peek Into the Reasoning of Neural Networks: Interpreting with Structural Visual Concepts
• Efficient Pre-trained Features and Recurrent Pseudo-Labeling in Unsupervised Domain Adaptation
• Light Multi-segment Activation for Model Compression
• The NiuTrans System for the WMT21 Efficiency Task
• Deep Unsupervised Drum Transcription
• Vision at A Glance: Interplay between Fine and Coarse Information Processing Pathways
• Split-and-Bridge: Adaptable Class Incremental Learning within a Single Neural Network
• Decision Support for Video-based Detection of Flu Symptoms
• Supervised Domain Enablement Attention for Personalized Domain Classification
• FastSal: a Computationally Efficient Network for Visual Saliency Prediction
• Fixed-point Factorized Networks
• The NiuTrans System for WNGT 2020 Efficiency Task
• Total Recall: a Customized Continual Learning Method for Neural Semantic Parsers
• Multi-Scale Aligned Distillation for Low-Resolution Detection
• A deep Natural Language Inference predictor without language-specific training data
• Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classification
• Learning Metrics from Mean Teacher: A Supervised Learning Method for Improving the Generalization of Speaker Verification System
• Exact and Consistent Interpretation of Piecewise Linear Models Hidden behind APIs: A Closed Form Solution
• SLADE: A Self-Training Framework For Distance Metric Learning
• Activation Map Adaptation for Effective Knowledge Distillation
• iLGaCo: Incremental Learning of Gait Covariate Factors
• Fully Synthetic Data Improves Neural Machine Translation with Knowledge Distillation
• Learning Fast Matching Models from Weak Annotations
• Interpretable Few-Shot Learning via Linear Distillation
• FLiText: A Faster and Lighter Semi-Supervised Text Classification with Convolution Networks
• Learning Sparse Structured Ensembles with SG-MCMC and Network Pruning
• Distilling Double Descent
• Go Small and Similar: A Simple Output Decay Brings Better Performance
• Transductive Learning for Textual Few-Shot Classification in API-based Embedding Models
• Mutual Teaching for Graph Convolutional Networks
• Expected Tight Bounds for Robust Training
• Intra-Utterance Similarity Preserving Knowledge Distillation for Audio Tagging
• Shakeout: A New Approach to Regularized Deep Neural Network Training
• Network Implosion: Effective Model Compression for ResNets via Static Layer Pruning and Retraining
• Marginal Utility Diminishes: Exploring the Minimum Knowledge for BERT Knowledge Distillation
• Mutually-aware Sub-Graphs Differentiable Architecture Search
• Self Supervision to Distillation for Long-Tailed Visual Recognition
• Class Interference Regularization
• Debiasing Methods in Natural Language Understanding Make Bias More Accessible
• AdaK-NER: An Adaptive Top-K Approach for Named Entity Recognition with Incomplete Annotations
• Avoiding Inference Heuristics in Few-shot Prompt-based Finetuning
• Diversified Mutual Learning for Deep Metric Learning
• Uncertainty Measures in Neural Belief Tracking and the Effects on Dialogue Policy Performance
• Eliminating Sentiment Bias for Aspect-Level Sentiment Classification with Unsupervised Opinion Extraction
• On the Efficiency of Subclass Knowledge Distillation in Classification Tasks
• Exact Backpropagation in Binary Weighted Networks with Group Weight Transformations
• Matching the Clinical Reality: Accurate OCT-Based Diagnosis From Few Labels
• QuantNet: Learning to Quantize by Learning within Fully Differentiable Framework
• Unsupervised model compression for multilayer bootstrap networks
• Description of Structural Biases and Associated Data in Sensor-Rich Environments
• Online Unsupervised Learning of Visual Representations and Categories
• Bootstrapped Meta-Learning
• Deep SIMBAD: Active Landmark-based Self-localization Using Ranking -based Scene Descriptor
• Fast Camera Image Denoising on Mobile GPUs with Deep Learning, Mobile AI 2021 Challenge: Report
• Compressing Facial Makeup Transfer Networks by Collaborative Distillation and Kernel Decomposition
• Class Incremental Online Streaming Learning
• Class-Discriminative CNN Compression
• Exploiting Web Images for Fine-Grained Visual Recognition by Eliminating Noisy Samples and Utilizing Hard Ones
• Dataset Culling: Towards Efficient Training Of Distillation-Based Domain Specific Models
• Variational Bayesian Sequence-to-Sequence Networks for Memory-Efficient Sign Language Translation
• Sketching Linear Classifiers over Data Streams
• Layer Pruning on Demand with Intermediate CTC
• MICIK: MIning Cross-Layer Inherent Similarity Knowledge for Deep Model Compression
• Ultrafast Video Attention Prediction with Coupled Knowledge Distillation
• Learning to see across Domains and Modalities
• Introducing the DOME Activation Functions
• Beyond Categorical Label Representations for Image Classification
• A Computer Vision Approach to Combat Lyme Disease
• Cross-Task Representation Learning for Anatomical Landmark Detection
• BAMSProd: A Step towards Generalizing the Adaptive Optimization Methods to Deep Binary Model
• Binary Neural Network for Speaker Verification
• On-target Adaptation
• No Answer is Better Than Wrong Answer: A Reflection Model for Document Level Machine Reading Comprehension
• A Novel Co-design Peta-scale Heterogeneous Cluster for Deep Learning Training
• 3D Dense Geometry-Guided Facial Expression Synthesis by Adversarial Learning
• Learning from Matured Dumb Teacher for Fine Generalization
• Distilling the Knowledge of Large-scale Generative Models into Retrieval Models for Efficient Open-domain Conversation
• Sequential Dynamic Decision Making with Deep Neural Nets on a Test-Time Budget
• Canoe : A System for Collaborative Learning for Neural Nets
• Auto-Split: A General Framework of Collaborative Edge-Cloud AI
• Implicit Regularization of Bregman Proximal Point Algorithm and Mirror Descent on Separable Data
• Revealing the Distributional Vulnerability of Discriminators by Implicit Generators
• Less is more: Faster and better music version identification with embedding distillation
• Learning Energy-Based Approximate Inference Networks for Structured Applications in NLP
• Compressing Deep Convolutional Neural Networks by Stacking Low-dimensional Binary Convolution Filters
• On the Variance of the Fisher Information for Deep Learning
• AE-Netv2: Optimization of Image Fusion Efficiency and Network Architecture
• Customizing Contextualized Language Models forLegal Document Reviews
• View Distillation with Unlabeled Data for Extracting Adverse Drug Effects from User-Generated Data
• Elbert: Fast Albert with Confidence-Window Based Early Exit
• ADA-Tucker: Compressing Deep Neural Networks via Adaptive Dimension Adjustment Tucker Decomposition
• Back to Square One: Superhuman Performance in Chutes and Ladders Through Deep Neural Networks and Tree Search
• Dep-$L_0$: Improving $L_0$-based Network Sparsification via Dependency Modeling
• Locally Linear Region Knowledge Distillation
• An Embedded Deep Learning based Word Prediction
• Knowledge Distillation By Sparse Representation Matching
• Reliable Identification of Redundant Kernels for Convolutional Neural Network Compression
• Deploying a BERT-based Query-Title Relevance Classifier in a Production System: a View from the Trenches
• Self-Regulation for Semantic Segmentation
• RefBERT: Compressing BERT by Referencing to Pre-computed Representations
• Learning Gradual Argumentation Frameworks using Genetic Algorithms
• CNN-based Semantic Segmentation using Level Set Loss
• Kernelized Classification in Deep Networks
• Towards Zero-Shot Knowledge Distillation for Natural Language Processing
• Small-Group Learning, with Application to Neural Architecture Search
• Joint Distribution across Representation Space for Out-of-Distribution Detection
• Transferring Knowledge with Attention Distillation for Multi-Domain Image-to-Image Translation
• BERT2DNN: BERT Distillation with Massive Unlabeled Data for Online E-Commerce Search
• Interpreting Attributions and Interactions of Adversarial Attacks
• KCP: Kernel Cluster Pruning for Dense Labeling Neural Networks
• Novel Adaptive Binary Search Strategy-First Hybrid Pyramid- and Clustering-Based CNN Filter Pruning Method without Parameters Setting
• Compression of Acoustic Event Detection Models With Quantized Distillation
• Environmental sound analysis with mixup based multitask learning and cross-task fusion
• Industry Scale Semi-Supervised Learning for Natural Language Understanding
• Mimicking Ensemble Learning with Deep Branched Networks
• Training CNNs faster with Dynamic Input and Kernel Downsampling
• Identifying and Exploiting Structures for Reliable Deep Learning
• 4-Connected Shift Residual Networks
• Generative Adversarial Network based Speaker Adaptation for High Fidelity WaveNet Vocoder
• OICSR: Out-In-Channel Sparsity Regularization for Compact Deep Neural Networks
• Incrementally Zero-Shot Detection by an Extreme Value Analyzer
• Unsupervised Local Discrimination for Medical Images
• A Comparative Study of Neural Network Compression
• Experimenting with Knowledge Distillation techniques for performing Brain Tumor Segmentation
• Discriminative Distillation to Reduce Class Confusion in Continual Learning
• Neighbourhood Distillation: On the benefits of non end-to-end distillation
• Large Product Key Memory for Pretrained Language Models
• Costs to Consider in Adopting NLP for Your Business
• A distillation based approach for the diagnosis of diseases
• Unsupervised Spoken Term Discovery on Untranscribed Speech
• Explore the Knowledge contained in Network Weights to Obtain Sparse Neural Networks
• Be Your Own Best Competitor! Multi-Branched Adversarial Knowledge Transfer
• Distilling particle knowledge for fast reconstruction at high-energy physics experiments
• Training Domain Specific Models for Energy-Efficient Object Detection
• Dual-Tuning: Joint Prototype Transfer and Structure Regularization for Compatible Feature Learning
• Boosting Summarization with Normalizing Flows and Aggressive Training
• Rethinking Uncertainty in Deep Learning: Whether and How it Improves Robustness
• LoANs: Weakly Supervised Object Detection with Localizer Assessor Networks
• Deeply Shared Filter Bases for Parameter-Efficient Convolutional Neural Networks
• Continual Learning for Image-Based Camera Localization
• Continual learning using hash-routed convolutional neural networks
• Feature-Supervised Action Modality Transfer
• Regularization via Adaptive Pairwise Label Smoothing
• Hands-on Guidance for Distilling Object Detectors
• Recurrent Stacking of Layers in Neural Networks: An Application to Neural Machine Translation
• FUNN: Flexible Unsupervised Neural Network
• Weakly Supervised Tracklet Person Re-Identification by Deep Feature-wise Mutual Learning
• Dynamic Domain Adaptation for Efficient Inference
• SSLayout360: Semi-Supervised Indoor Layout Estimation from 360-Degree Panorama
• Weighted Ensemble Models Are Strong Continual Learners
• Unsupervised Domain Adaptive Person Re-Identification via Human Learning Imitation
• Midpoint Regularization: from High Uncertainty Training to Conservative Classification
• Deep Virtual Networks for Memory Efficient Inference of Multiple Tasks
• Adaptive Explainable Continual Learning Framework for Regression Problems with Focus on Power Forecasts
• Mobile Recognition of Wikipedia Featured Sites using Deep Learning and Crowd-sourced Imagery
• DNN Quantization with Attention
• Joint Learning for Scattered Point Cloud Understanding with Hierarchical Self-Distillation
• Towards Explainable Fact Checking
• CoCo DistillNet: a Cross-layer Correlation Distillation Network for Pathological Gastric Cancer Segmentation
• Distilling Interpretable Models into Human-Readable Code
• A Generalized Meta-loss function for regression and classification using privileged information
• CDN-MEDAL: Two-stage Density and Difference Approximation Framework for Motion Analysis
• DecisiveNets: Training Deep Associative Memories to Solve Complex Machine Learning Problems
• Single Image Depth Prediction with Wavelet Decomposition
• Better Knowledge Retention through Metric Learning
• Knowledge Distillation from BERT Transformer to Speech Transformer for Intent Classification
• Decoupled Transformer for Scalable Inference in Open-domain Question Answering
• Towards Making Deep Transfer Learning Never Hurt
• Filter Pre-Pruning for Improved Fine-tuning of Quantized Deep Neural Networks
• Effective Model Compression via Stage-wise Pruning
• WeClick: Weakly-Supervised Video Semantic Segmentation with Click Annotations
• Learning Compatible Embeddings
• Incremental Knowledge Based Question Answering
• Cogradient Descent for Dependable Learning
• Generative Low-Shot Network Expansion
• Network-Agnostic Knowledge Transfer for Medical Image Segmentation
• CNN inference acceleration using dictionary of centroids
• ZS-IL: Looking Back on Learned Experiences For Zero-Shot Incremental Learning
• Sequence-level self-learning with multiple hypotheses
• Training compact deep learning models for video classification using circulant matrices
• C2S2: Cost-aware Channel Sparse Selection for Progressive Network Pruning
• ROSITA: Refined BERT cOmpreSsion with InTegrAted techniques
• Inverse Problems Leveraging Pre-trained Contrastive Representations
• A Novel Unsupervised Post-Processing Calibration Method for DNNS with Robustness to Domain Shift
• Parallel Blockwise Knowledge Distillation for Deep Neural Network Compression
• Inferring Mesoscale Models of Neural Computation
• Decision Theoretic Bootstrapping
• Radius Adaptive Convolutional Neural Network
• Scene-adaptive Knowledge Distillation for Sequential Recommendation via Differentiable Architecture Search
• GRIm-RePR: Prioritising Generating Important Features for Pseudo-Rehearsal
• Transfer Value Iteration Networks
• CNN with large memory layers
• Intra-Model Collaborative Learning of Neural Networks
• Self-Supervised Adaptation for Video Super-Resolution
• Investigations on the inference optimization techniques and their impact on multiple hardware platforms for Semantic Segmentation
• Reciprocal Supervised Learning Improves Neural Machine Translation
• Reducing the Deployment-Time Inference Control Costs of Deep Reinforcement Learning Agents via an Asymmetric Architecture
• AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language Models
• Lifelong Spectral Clustering
• Student-Teacher Learning from Clean Inputs to Noisy Inputs
• Adversarial Attacks on Cognitive Self-Organizing Networks: The Challenge and the Way Forward
• Incorporating a Local Translation Mechanism into Non-autoregressive Translation
• Discriminative, Generative and Self-Supervised Approaches for Target-Agnostic Learning
• Zero Training Overhead Portfolios for Learning to Solve Combinatorial Problems
• MDQE: A More Accurate Direct Pretraining for Machine Translation Quality Estimation
• Class-incremental Learning with Rectified Feature-Graph Preservation
• Fast Intent Classification for Spoken Language Understanding
• Implicit Priors for Knowledge Sharing in Bayesian Neural Networks
• Efficient Fusion of Sparse and Complementary Convolutions
• MED-TEX: Transferring and Explaining Knowledge with Less Data from Pretrained Medical Imaging Models
• Compensation Learning
• PnPOOD : Out-Of-Distribution Detection for Text Classification via Plug andPlay Data Augmentation
• On Estimating the Training Cost of Conversational Recommendation Systems
• Do Not Forget to Attend to Uncertainty while Mitigating Catastrophic Forgetting
• Low-Rank+Sparse Tensor Compression for Neural Networks
• Domain-Agnostic Clustering with Self-Distillation
• Dual-Teacher++: Exploiting Intra-domain and Inter-domain Knowledge with Reliable Transfer for Cardiac Segmentation
• Knowledge Distillation of a Protein Language Model Yields a Foundational Implicit Solvent Model
• Re-Evaluating GermEval17 Using German Pre-Trained Language Models
• Improving the expressiveness of neural vocoding with non-affine Normalizing Flows
• jBOT: Semantic Jet Representation Clustering Emerges from Self-Distillation
• Label Denoising with Large Ensembles of Heterogeneous Neural Networks
• Designing a Minimal Retrieve-and-Read System for Open-Domain Question Answering
• Conditional Attention Networks for Distilling Knowledge Graphs in Recommendation
• Real-time Surgical Environment Enhancement for Robot-Assisted Minimally Invasive Surgery Based on Super-Resolution
• Revisiting the Importance of Encoding Logic Rules in Sentiment Classification
• Magnitude and Uncertainty Pruning Criterion for Neural Networks
• Hardware-efficient Residual Networks for FPGAs
• Evolutionary Hyperparameter Optimization to Find Lightweight CNN Models for Autonomous Steering
• Human-Like Active Learning: Machines Simulating the Human Learning Process
• Collaborative Group Learning
• A Quadratic Actor Network for Model-Free Reinforcement Learning
• Ensemble Knowledge Distillation for CTR Prediction
• Iterative Dual Domain Adaptation for Neural Machine Translation
• Constrained-size Tensorflow Models for YouTube-8M Video Understanding Challenge
• Secure Machine Learning in the Cloud Using One Way Scrambling by Deconvolution
• Building Deep Learning Models to Predict Mortality in ICU Patients
• SlimNets: An Exploration of Deep Model Compression and Acceleration
• Bridging the gap between Human Action Recognition and Online Action Detection
• Sidekick Policy Learning for Active Visual Exploration
• Interpretable Attention Guided Network for Fine-grained Visual Classification
• AUTOKD: Automatic Knowledge Distillation Into A Student Architecture Family
• Graphs for deep learning representations
• GHFP: Gradually Hard Filter Pruning
• Cross-Modality Distillation: A case for Conditional Generative Adversarial Networks
• Self-supervised Knowledge Distillation Using Singular Value Decomposition
• Weight Distillation: Transferring the Knowledge in Neural Network Parameters
• Accuracy to Throughput Trade-offs for Reduced Precision Neural Networks on Reconfigurable Logic
• Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question-Answering Data
• How Does Distilled Data Complexity Impact the Quality and Confidence of Non-Autoregressive Machine Translation?
• Class-Incremental Domain Adaptation with Smoothing and Calibration for Surgical Report Generation
• Towards Enabling Meta-Learning from Target Models
• Modeling Musical Structure with Artificial Neural Networks
• Achieving Adversarial Robustness Requires An Active Teacher
• Learning Realistic Patterns from Unrealistic Stimuli: Generalization and Data Anonymization
• An Improving Framework of regularization for Network Compression
• Coded-InvNet for Resilient Prediction Serving Systems
• Using Under-trained Deep Ensembles to Learn Under Extreme Label Noise
• An Ensemble with Shared Representations Based on Convolutional Networks for Continually Learning Facial Expressions
• BRIEF: Backward Reduction of CNNs with Information Flow Analysis
• Using Cross-Model EgoSupervision to Learn Cooperative Basketball Intention
• Representation Evaluation Block-based Teacher-Student Network for the Industrial Quality-relevant Performance Modeling and Monitoring
• Towards Quantifiable Dialogue Coherence Evaluation
• Teach an all-rounder with experts in different domains
• SmartDeal: Re-Modeling Deep Network Weights for Efficient Inference and Training
• Masked Training of Neural Networks with Partial Gradients
• MSD: Saliency-aware Knowledge Distillation for Multimodal Understanding
• BridgeNets: Student-Teacher Transfer Learning Based on Recursive Neural Networks and its Application to Distant Speech Recognition
• Distilling Image Classifiers in Object Detectors
• Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting
• What classifiers know what they don't?
• SVMax: A Feature Embedding Regularizer
• Student Helping Teacher: Teacher Evolution via Self-Knowledge Distillation
• Single Model Ensemble using Pseudo-Tags and Distinct Vectors
• Technical Report of Team GraphMIRAcles in the WikiKG90M-LSC Track of OGB-LSC @ KDD Cup 2021
• Structured Ensembles: an Approach to Reduce the Memory Footprint of Ensemble Methods
• Knowledge distillation from multi-modal to mono-modal segmentation networks
• Alleviating Noisy-label Effects in Image Classification via Probability Transition Matrix
• Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimization
• Can a powerful neural network be a teacher for a weaker neural network?
• Multilingual AMR Parsing with Noisy Knowledge Distillation
• GenURL: A General Framework for Unsupervised Representation Learning
• Scalable Consistency Training for Graph Neural Networks via Self-Ensemble Self-Distillation
• A More Efficient Chinese Named Entity Recognition base on BERT and Syntactic Analysis
• Carrying out CNN Channel Pruning in a White Box
• Bayesian Sparsification Methods for Deep Complex-valued Networks
• Hardware-Software Codesign of Accurate, Multiplier-free Deep Neural Networks
• Block-term Tensor Neural Networks
• Towards an Understanding of Default Policies in Multitask Policy Optimization
• Is the Meta-Learning Idea Able to Improve the Generalization of Deep Neural Networks on the Standard Supervised Learning?
• Autoregressive Knowledge Distillation through Imitation Learning
• Dark Reciprocal-Rank: Boosting Graph-Convolutional Self-Localization Network via Teacher-to-student Knowledge Transfer
• Automatic low-bit hybrid quantization of neural networks through meta learning
• Global Structure-Aware Drum Transcription Based on Self-Attention Mechanisms
• Analyzing the benefits of communication channels between deep learning models
• TAGLETS: A System for Automatic Semi-Supervised Learning with Auxiliary Data
• Coarse and fine-grained automatic cropping deep convolutional neural network
• MicroNet for Efficient Language Modeling
• CP-NAS: Child-Parent Neural Architecture Search for Binary Neural Networks
• Learning Proposals for Practical Energy-Based Regression
• L-CO-Net: Learned Condensation-Optimization Network for Clinical Parameter Estimation from Cardiac Cine MRI
• Noise Stability Regularization for Improving BERT Fine-tuning
• HRKD: Hierarchical Relational Knowledge Distillation for Cross-domain Language Model Compression
• $S^2$-LBI: Stochastic Split Linearized Bregman Iterations for Parsimonious Deep Learning
• Lifelong Mixture of Variational Autoencoders
• Incremental Class Learning using Variational Autoencoders with Similarity Learning
• CHEER: Rich Model Helps Poor Model via Knowledge Infusion
• EZCrop: Energy-Zoned Channels for Robust Output Pruning
• Using Machine Translation to Localize Task Oriented NLG Output
• Pseudo Labeling and Negative Feedback Learning for Large-scale Multi-label Domain Classification
• Semi-supervised Acoustic Event Detection based on tri-training
• Selective Convolutional Network: An Efficient Object Detector with Ignoring Background
• Building effective deep neural network architectures one feature at a time
• Joint Text and Label Generation for Spoken Language Understanding
• Deep Cerebellar Nuclei Segmentation via Semi-Supervised Deep Context-Aware Learning from 7T Diffusion MRI
• ATSO: Asynchronous Teacher-Student Optimization for Semi-Supervised Medical Image Segmentation
• Training Over-parameterized Models with Non-decomposable Objectives
• Condensing Two-stage Detection with Automatic Object Key Part Discovery
• Triplet Loss for Knowledge Distillation
• Pacemaker: Intermediate Teacher Knowledge Distillation For On-The-Fly Convolutional Neural Network
• Learning a No-Reference Quality Assessment Model of Enhanced Images With Big Data
• Neuroevolutionary Transfer Learning of Deep Recurrent Neural Networks through Network-Aware Adaptation
• Multi-Glimpse Network: A Robust and Efficient Classification Architecture based on Recurrent Downsampled Attention
• Domain Adaptation Using Class Similarity for Robust Speech Recognition
• Adversarially Robust and Explainable Model Compression with On-Device Personalization for Text Classification
• Unlabeled Data Deployment for Classification of Diabetic Retinopathy Images Using Knowledge Transfer
• FoCL: Feature-Oriented Continual Learning for Generative Models
• MultiOpEd: A Corpus of Multi-Perspective News Editorials
• Using an ensemble color space model to tackle adversarial examples
• CARLS: Cross-platform Asynchronous Representation Learning System
• Sequence-to-Sequence Learning via Attention Transfer for Incremental Speech Recognition
• JIT-Masker: Efficient Online Distillation for Background Matting
• Divergent Search for Few-Shot Image Classification
• The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation
• On Cross-Layer Alignment for Model Fusion of Heterogeneous Neural Networks
• Recurrent Distillation based Crowd Counting
• Risk factor identification for incident heart failure using neural network distillation and variable selection
• Deep Model Compression Via Two-Stage Deep Reinforcement Learning
• Attention to detail: inter-resolution knowledge distillation
• Integrating Knowledge into End-to-End Speech Recognition from External Text-Only Data
• Inter-Domain Alignment for Predicting High-Resolution Brain Networks Using Teacher-Student Learning
• A Survey on Green Deep Learning
• Beyond without Forgetting: Multi-Task Learning for Classification with Disjoint Datasets
• Residual Continual Learning
• Towards Robust Classification with Image Quality Assessment
• Understanding Robustness in Teacher-Student Setting: A New Perspective
• Feature Lenses: Plug-and-play Neural Modules for Transformation-Invariant Visual Representations
• Meta-Teacher For Face Anti-Spoofing
• Sequential Feature Filtering Classifier
• Perceptual Gradient Networks
• Target Layer Regularization for Continual Learning Using Cramer-Wold Generator
• Weakly-Supervised Spatial Context Networks
• Information Fusion in Attention Networks Using Adaptive and Multi-level Factorized Bilinear Pooling for Audio-visual Emotion Recognition
• Alpha-Net: Architecture, Models, and Applications
• Stochastic Batch Augmentation with An Effective Distilled Dynamic Soft Label Regularizer
• Semantic Relation Preserving Knowledge Distillation for Image-to-Image Translation
• Towards Data-Free Domain Generalization
• Domain-Specific Bias Filtering for Single Labeled Domain Generalization
• Sub-bit Neural Networks: Learning to Compress and Accelerate Binary Neural Networks
• Density Fixing: Simple yet Effective Regularization Method based on the Class Prior
• GIM: Gaussian Isolation Machines
• Attention-Free Keyword Spotting
• Confidence Preservation Property in Knowledge Distillation Abstractions
• RSAC: Regularized Subspace Approximation Classifier for Lightweight Continuous Learning
• Toward Compact Parameter Representations for Architecture-Agnostic Neural Network Compression
• Graph Partner Neural Networks for Semi-Supervised Learning on Graphs
• BERMo: What can BERT learn from ELMo?
• Exploring Knowledge Distillation of a Deep Neural Network for Multi-Script identification
• Sparse Architectures for Text-Independent Speaker Verification Using Deep Neural Networks
• Toxicity Detection can be Sensitive to the Conversational Context
• Impression Space from Deep Template Network
• Understanding Interpretability by generalized distillation in Supervised Classification
• Provable Lifelong Learning of Representations
• Hierarchical Knowledge Distillation for Dialogue Sequence Labeling
• Topology Distillation for Recommender System
• Powering COVID-19 community Q&A with Curated Side Information
• Network Adjustment: Channel Search Guided by FLOPs Utilization Ratio
• Dual-Teacher: Integrating Intra-domain and Inter-domain Teachers for Annotation-efficient Cardiac Segmentation
• A copula-based visualization technique for a neural network
• Multi-Level Transfer Learning from Near-Field to Far-Field Speaker Verification
• Generating Video Descriptions with Topic Guidance
• Semi-Supervised Learning with Taxonomic Labels
• Improving Word Embedding Factorization for Compression Using Distilled Nonlinear Neural Decomposition
• Channel Planting for Deep Neural Networks using Knowledge Distillation
• Can We Teach Computers to Understand Art? Domain Adaptation for Enhancing Deep Networks Capacity to De-Abstract Art
• Differential Replication in Machine Learning
• Layer-Wise Multi-View Learning for Neural Machine Translation
• EvDistill: Asynchronous Events to End-task Learning via Bidirectional Reconstruction-guided Cross-modal Knowledge Distillation
• Learning Dynamic BERT via Trainable Gate Variables and a Bi-modal Regularizer
• HAWKEYE: Adversarial Example Detector for Deep Neural Networks
• Out-of-Category Document Identification Using Target-Category Names as Weak Supervision
• Visualizing Classification Structure of Large-Scale Classifiers
• Coarse-To-Fine Incremental Few-Shot Learning
• Exploring Low-Cost Transformer Model Compression for Large-Scale Commercial Reply Suggestions
• Balancing Accuracy and Latency in Multipath Neural Networks
• Combining Different V1 Brain Model Variants to Improve Robustness to Image Corruptions in CNNs
• Low-bit Quantization of Recurrent Neural Network Language Models Using Alternating Direction Methods of Multipliers
• MC-SSL0.0: Towards Multi-Concept Self-Supervised Learning
• ReadOnce Transformers: Reusable Representations of Text for Transformers
• A Knowledge Transfer Framework for Differentially Private Sparse Learning
• Training BatchNorm Only in Neural Architecture Search and Beyond
• Knowledge-Adaptation Priors
• Fine-grained Knowledge Fusion for Sequence Labeling Domain Adaptation
• Online Filter Clustering and Pruning for Efficient Convnets
• Self-Teaching Networks
• A Highly Effective Low-Rank Compression of Deep Neural Networks with Modified Beam-Search and Modified Stable Rank
• When in Doubt, Summon the Titans: Efficient Inference with Large Models
• A Note on Knowledge Distillation Loss Function for Object Classification
• Resource-Efficient Speech Mask Estimation for Multi-Channel Speech Enhancement
• Subword Language Model for Query Auto-Completion
• RGB-D Salient Object Detection Based on Discriminative Cross-modal Transfer Learning
• Efficient Transfer Learning via Joint Adaptation of Network Architecture and Weight
• Regression Bugs Are In Your Model! Measuring, Reducing and Analyzing Regressions In NLP Model Updates
• SDM-Net: A Simple and Effective Model for Generalized Zero-Shot Learning
• Saliency Prediction with External Knowledge
• Learning Task-oriented Disentangled Representations for Unsupervised Domain Adaptation
• When Stability meets Sufficiency: Informative Explanations that do not Overwhelm
• Energy Clustering for Unsupervised Person Re-identification
• Does Thermal data make the detection systems more reliable?
• Contraction Mapping of Feature Norms for Classifier Learning on the Data with Different Quality
• A Note on Latency Variability of Deep Neural Networks for Mobile Inference
• Confidence Conditioned Knowledge Distillation
• Are Compressed Language Models Less Subgroup Robust?
• Weight Squeezing: Reparameterization for Knowledge Transfer and Model Compression
• How many images do I need? Understanding how sample size per class affects deep learning model performance metrics for balanced designs in autonomous wildlife monitoring
• Learning Rich Nearest Neighbor Representations from Self-supervised Ensembles
• Modularized Transfomer-based Ranking Framework
• Long Short-Term Sample Distillation
• Isotonic Data Augmentation for Knowledge Distillation
• Semi-Supervised Text Classification via Self-Pretraining
• Exploring Teacher-Student Learning Approach for Multi-lingual Speech-to-Intent Classification
• k-Nearest Neighbors by Means of Sequence to Sequence Deep Neural Networks and Memory Networks
• Metric-based Regularization and Temporal Ensemble for Multi-task Learning using Heterogeneous Unsupervised Tasks
• All Word Embeddings from One Embedding
• Multi-Layer Softmaxing during Training Neural Machine Translation for Flexible Decoding with Fewer Layers
• Online Sensor Hallucination via Knowledge Distillation for Multimodal Image Classification
• The pursuit of beauty: Converting image labels to meaningful vectors
• Towards Real-time Mispronunciation Detection in Kids' Speech
• Deep Epidemiological Modeling by Black-box Knowledge Distillation: An Accurate Deep Learning Model for COVID-19
• Skimming and Scanning for Untrimmed Video Action Recognition
• Tensor Methods for Generating Compact Uncertainty Quantification and Deep Learning Models
• Efficient Federated Learning for AIoT Applications Using Knowledge Distillation
• Image-based OoD-Detector Principles on Graph-based Input Data in Human Action Recognition
• Dual Correction Strategy for Ranking Distillation in Top-N Recommender System
• Automatic Documentation of ICD Codes with Far-Field Speech Recognition
• Partial to Whole Knowledge Distillation: Progressive Distilling Decomposed Knowledge Boosts Student Better
• Jointly Learning to Repair Code and Generate Commit Message
• Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference
• Distiller: A Systematic Study of Model Distillation Methods in Natural Language Processing
• Unsupervised Paraphrasing with Pretrained Language Models
• How to Explain Neural Networks: an Approximation Perspective
• Cooperative Bi-path Metric for Few-shot Learning
• Dynamic Knowledge Distillation for Pre-trained Language Models
• Weakly-Supervised Monocular Depth Estimationwith Resolution-Mismatched Data
• Simulated Chats for Building Dialog Systems: Learning to Generate Conversations from Instructions
• K-AID: Enhancing Pre-trained Language Models with Domain Knowledge for Question Answering
• CoRe: An Efficient Coarse-refined Training Framework for BERT
• Distilling Knowledge From a Deep Pose Regressor Network
• Low-Latency Incremental Text-to-Speech Synthesis with Distilled Context Prediction Network
• IB-DRR: Incremental Learning with Information-Back Discrete Representation Replay
• Towards Accurate Knowledge Transfer via Target-awareness Representation Disentanglement
• TEAM: We Need More Powerful Adversarial Examples for DNNs
• Transferring Inter-Class Correlation
• Learning to Transfer Learn: Reinforcement Learning-Based Selection for Adaptive Transfer Learning
• New Perspective on Progressive GANs Distillation for One-class Novelty Detection
• Rethnicity: Predicting Ethnicity from Names
• Towards Modality Transferable Visual Information Representation with Optimal Model Compression
• CompConv: A Compact Convolution Module for Efficient Feature Learning
• Knowledge Distillation with Noisy Labels for Natural Language Understanding
• Denoising and Verification Cross-Layer Ensemble Against Black-box Adversarial Attacks
• Multiple Policy Value Monte Carlo Tree Search
• Homography augumented momentum constrastive learning for SAR image retrieval
• Filter Grafting for Deep Neural Networks: Reason, Method, and Cultivation
• Model Composition: Can Multiple Neural Networks Be Combined into a Single Network Using Only Unlabeled Data?
• A Studious Approach to Semi-Supervised Learning
• Distilling Linguistic Context for Language Model Compression
• FastMVAE2: On improving and accelerating the fast variational autoencoder-based source separation algorithm for determined mixtures
• Classification-based Quality Estimation: Small and Efficient Models for Real-world Applications
• Association: Remind Your GAN not to Forget
• MixMix: All You Need for Data-Free Compression Are Feature and Data Mixing
• Annealing Knowledge Distillation
• OBoW: Online Bag-of-Visual-Words Generation for Self-Supervised Learning
• Layer Reduction: Accelerating Conformer-Based Self-Supervised Model via Layer Consistency
• Leveraging Acoustic and Linguistic Embeddings from Pretrained speech and language Models for Intent Classification
• Learn to Talk via Proactive Knowledge Transfer
• EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation