CondConv: Conditionally Parameterized Convolutions for Efficient Inference

Abstract

Convolutional layers are one of the basic building blocks of modern deep neural networks. One fundamental assumption is that convolutional kernels should be shared for all examples in a dataset. We propose conditionally parameterized convolutions (CondConv), which learn specialized convolutional kernels for each example. Replacing normal convolutions with CondConv enables us to increase the size and capacity of a network, while maintaining efficient inference. We demonstrate that scaling networks with CondConv improves the performance and inference cost trade-off of several existing convolutional neural network architectures on both classification and detection tasks. On ImageNet classification, our CondConv approach applied to EfficientNet-B0 achieves state-of-the-art performance of 78.3% accuracy with only 413M multiply-adds. Code and checkpoints for the CondConv Tensorflow layer and CondConv-EfficientNet models are available at: https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet/condconv.

Cited by in corpus (62)

• Attention Mechanisms in Computer Vision: A Survey
• Bringing AI To Edge: From Deep Learning's Perspective
• Dynamic Convolution: Attention over Convolution Kernels
• DyNet: Dynamic Convolution for Accelerating Convolutional Neural Networks
• NBDT: Neural-Backed Decision Trees
• Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition
• Dynamic Neural Networks: A Survey
• MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers
• TAda! Temporally-Adaptive Convolutions for Video Understanding
• Scaling Vision with Sparse Mixture of Experts
• PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Clouds
• Conditional Convolutions for Instance Segmentation
• Involution: Inverting the Inherence of Convolution for Visual Recognition
• Diverse Branch Block: Building a Convolution as an Inception-like Unit
• TAM: Temporal Adaptive Module for Video Recognition
• AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognition
• MicroNet: Towards Image Recognition with Extremely Low FLOPs
• CondLaneNet: a Top-to-down Lane Detection Framework Based on Conditional Convolution
• Revisiting Dynamic Convolution via Matrix Decomposition
• Lite-HRNet: A Lightweight High-Resolution Network
• Revisiting Spatial Invariance with Low-Rank Local Connectivity
• Activate or Not: Learning Customized Activation
• Dynamic Region-Aware Convolution
• DoDNet: Learning to segment multi-organ and tumors from multiple partially labeled datasets
• Fighting Gradients with Gradients: Dynamic Defenses against Adversarial Attacks
• Learning A Single Network for Scale-Arbitrary Super-Resolution
• Collaboration of Experts: Achieving 80% Top-1 Accuracy on ImageNet with 100M FLOPs
• Domain-Aware Dynamic Networks
• Conditional Directed Graph Convolution for 3D Human Pose Estimation
• Conditionally Parameterized, Discretization-Aware Neural Networks for Mesh-Based Modeling of Physical Systems
• Instance and Panoptic Segmentation Using Conditional Convolutions
• WeightNet: Revisiting the Design Space of Weight Networks
• DS-Net++: Dynamic Weight Slicing for Efficient Inference in CNNs and Transformers
• High-Capacity Expert Binary Networks
• MicroNet: Improving Image Recognition with Extremely Low FLOPs
• Learning Affinity-Aware Upsampling for Deep Image Matting
• FCPose: Fully Convolutional Multi-Person Pose Estimation with Dynamic Instance-Aware Convolutions
• Synthetic Aperture Radar Image Change Detection via Siamese Adaptive Fusion Network
• InsPose: Instance-Aware Networks for Single-Stage Multi-Person Pose Estimation
• Dynamic Image Restoration and Fusion Based on Dynamic Degradation
• LAConv: Local Adaptive Convolution for Image Fusion
• Distractor-Aware Fast Tracking via Dynamic Convolutions and MOT Philosophy
• Shift-and-Balance Attention
• Dynamic Graph: Learning Instance-aware Connectivity for Neural Networks
• Instance and Pair-Aware Dynamic Networks for Re-Identification
• PointINS: Point-based Instance Segmentation
• Decoupled Dynamic Filter Networks
• Temporal Dynamic Convolutional Neural Network for Text-Independent Speaker Verification and Phonemetic Analysis
• Image Synthesis via Semantic Composition
• Towards Understanding the Effectiveness of Attention Mechanism
• BasisNet: Two-stage Model Synthesis for Efficient Inference
• Dynamic Slimmable Network
• Going Beyond Saliency Maps: Training Deep Models to Interpret Deep Models
• Deeply Shared Filter Bases for Parameter-Efficient Convolutional Neural Networks
• CoCAtt: A Cognitive-Conditioned Driver Attention Dataset
• Dynamic Normalization
• Dynamic Transfer for Multi-Source Domain Adaptation
• DRDF: Determining the Importance of Different Multimodal Information with Dual-Router Dynamic Framework
• Dynamic Parameterized Network for CTR Prediction
• Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference
• Context-Aware Dynamic Feature Extraction for 3D Object Detection in Point Clouds
• Handling Variable-Dimensional Time Series with Graph Neural Networks