Convolutional Neural Fabrics
arXiv:1606.02492
Abstract
Despite the success of CNNs, selecting the optimal architecture for a given task remains an open problem. Instead of aiming to select a single optimal architecture, we propose a "fabric" that embeds an exponentially large number of architectures. The fabric consists of a 3D trellis that connects response maps at different layers, scales, and channels with a sparse homogeneous local connectivity pattern. The only hyper-parameters of a fabric are the number of channels and layers. While individual architectures can be recovered as paths, the fabric can in addition ensemble all embedded architectures together, sharing their weights where their paths overlap. Parameters can be learned using standard methods based on back-propagation, at a cost that scales linearly in the fabric size. We present benchmark results competitive with the state of the art for image classification on MNIST and CIFAR10, and for semantic segmentation on the Part Labels dataset.
Corrected typos (In proceedings of NIPS16 )
References in corpus (4)
Cited by in corpus (42)
- Neural Architecture Search: A Survey
- AutoML: A Survey of the State-of-the-Art
- PathNet: Evolution Channels Gradient Descent in Super Neural Networks
- Deep High-Resolution Representation Learning for Visual Recognition
- A Comprehensive Analysis of Deep Regression
- Understanding and Robustifying Differentiable Architecture Search
- A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions
- Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search
- Big-Little Net: An Efficient Multi-Scale Feature Representation for Visual and Speech Recognition
- NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Search
- A Closer Look at Accuracy vs. Robustness
- Efficient Multi-objective Neural Architecture Search via Lamarckian Evolution
- On Characterizing the Capacity of Neural Networks using Algebraic Topology
- MaskConnect: Connectivity Learning by Gradient Descent
- Analysis and Optimization of Convolutional Neural Network Architectures
- Improving Auto-Augment via Augmentation-Wise Weight Sharing
- Practical Block-wise Neural Network Architecture Generation
- BlockQNN: Efficient Block-wise Neural Network Architecture Generation
- AM-LFS: AutoML for Loss Function Search
- Pose Neural Fabrics Search
- SGAS: Sequential Greedy Architecture Search
- A Framework for Searching for General Artificial Intelligence
- Anytime Stereo Image Depth Estimation on Mobile Devices
- Nanophotonic Media for Artificial Neural Inference
- Optimization Algorithm Inspired Deep Neural Network Structure Design
- Online Hyper-parameter Learning for Auto-Augmentation Strategy
- MTL-NAS: Task-Agnostic Neural Architecture Search towards General-Purpose Multi-Task Learning
- IRLAS: Inverse Reinforcement Learning for Architecture Search
- ImmuNeCS: Neural Committee Search by an Artificial Immune System
- Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias
- Fast Neural Architecture Construction using EnvelopeNets
- AutoGrow: Automatic Layer Growing in Deep Convolutional Networks
- NAS-Count: Counting-by-Density with Neural Architecture Search
- FlatteNet: A Simple Versatile Framework for Dense Pixelwise Prediction
- Doubly Nested Network for Resource-Efficient Inference
- EfficientHRNet: Efficient Scaling for Lightweight High-Resolution Multi-Person Pose Estimation
- Optimizing Neural Architecture Search using Limited GPU Time in a Dynamic Search Space: A Gene Expression Programming Approach
- Understanding the Effects of Pre-Training for Object Detectors via Eigenspectrum
- MS-DARTS: Mean-Shift Based Differentiable Architecture Search
- SparseMask: Differentiable Connectivity Learning for Dense Image Prediction
- GPNAS: A Neural Network Architecture Search Framework Based on Graphical Predictor
- Layer Flexible Adaptive Computational Time