Dynamic Filter Networks

Abstract

In a traditional convolutional layer, the learned filters stay fixed after training. In contrast, we introduce a new framework, the Dynamic Filter Network, where filters are generated dynamically conditioned on an input. We show that this architecture is a powerful one, with increased flexibility thanks to its adaptive nature, yet without an excessive increase in the number of model parameters. A wide variety of filtering operations can be learned this way, including local spatial transformations, but also others like selective (de)blurring or adaptive feature extraction. Moreover, multiple such layers can be combined, e.g. in a recurrent architecture. We demonstrate the effectiveness of the dynamic filter network on the tasks of video and stereo prediction, and reach state-of-the-art performance on the moving MNIST dataset with a much smaller model. By visualizing the learned filters, we illustrate that the network has picked up flow information by only looking at unlabelled training data. This suggests that the network can be used to pretrain networks for various supervised tasks in an unsupervised way, like optical flow and depth estimation.

submitted to NIPS16; X. Jia and B. De Brabandere contributed equally to this work and are listed in alphabetical order

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• Training Very Deep Networks
• Action-Conditional Video Prediction using Deep Networks in Atari Games
• Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis
• Spatio-temporal video autoencoder with differentiable memory
• DeepStereo: Learning to Predict New Views from the World's Imagery
• Deep3D: Fully Automatic 2D-to-3D Video Conversion with Deep Convolutional Neural Networks

Cited by in corpus (32)

• A Review on Deep Learning Techniques for Video Prediction
• Video Coding for Machines: A Paradigm of Collaborative Compression and Intelligent Analytics
• Iterative Filter Adaptive Network for Single Image Defocus Deblurring
• Robust sound event detection in bioacoustic sensor networks
• Dynamic Fusion Module Evolves Drivable Area and Road Anomaly Detection: A Benchmark and Algorithms
• Dynamic Instance Domain Adaptation
• Learning Spatial and Spatio-Temporal Pixel Aggregations for Image and Video Denoising
• Salient Object Detection via Dynamic Scale Routing
• Distortion-aware Monocular Depth Estimation for Omnidirectional Images
• End-to-end Learning of Driving Models from Large-scale Video Datasets
• Visual Dynamics: Stochastic Future Generation via Layered Cross Convolutional Networks
• Optical Flow Reusing for High-Efficiency Space-Time Video Super Resolution
• Policy Manifold Search: Exploring the Manifold Hypothesis for Diversity-based Neuroevolution
• Adaptive Filters in Graph Convolutional Neural Networks
• Predicting Video with VQVAE
• Enhanced Standard Compatible Image Compression Framework based on Auxiliary Codec Networks
• RDFNet: Regional Dynamic FISTA-Net for Spectral Snapshot Compressive Imaging
• Joint Learning Content and Degradation Aware Feature for Blind Super-Resolution
• Dynamic Refinement Network for Oriented and Densely Packed Object Detection
• Reduced-Gate Convolutional LSTM Using Predictive Coding for Spatiotemporal Prediction
• Adaptive Model Learning of Neural Networks with UUB Stability for Robot Dynamic Estimation
• CARAFE++: Unified Content-Aware ReAssembly of FEatures
• BLADE: Box-Level Supervised Amodal Segmentation through Directed Expansion
• High-efficient Bloch simulation of magnetic resonance imaging sequences based on deep learning
• Learning to Predict Context-adaptive Convolution for Semantic Segmentation
• L3DOC: Lifelong 3D Object Classification
• Optimising for Interpretability: Convolutional Dynamic Alignment Networks
• One Self-Configurable Model to Solve Many Abstract Visual Reasoning Problems
• WeightNet: Revisiting the Design Space of Weight Networks
• InsPose: Instance-Aware Networks for Single-Stage Multi-Person Pose Estimation
• Meta Matrix Factorization for Federated Rating Predictions
• An Efficient Deep Template Matching and In-Plane Pose Estimation Method via Template-Aware Dynamic Convolution