Resnet in Resnet: Generalizing Residual Architectures

Abstract

Residual networks (ResNets) have recently achieved state-of-the-art on challenging computer vision tasks. We introduce Resnet in Resnet (RiR): a deep dual-stream architecture that generalizes ResNets and standard CNNs and is easily implemented with no computational overhead. RiR consistently improves performance over ResNets, outperforms architectures with similar amounts of augmentation on CIFAR-10, and establishes a new state-of-the-art on CIFAR-100.

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Striving for Simplicity: The All Convolutional Net
• Training Very Deep Networks
• Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
• Grid Long Short-Term Memory
• Learning Longer Memory in Recurrent Neural Networks

Cited by in corpus (41)

• Densely Connected Convolutional Networks
• Density estimation using Real NVP
• FractalNet: Ultra-Deep Neural Networks without Residuals
• Residual Networks of Residual Networks: Multilevel Residual Networks
• Recent Advances in Convolutional Neural Networks
• Recent Advances in Deep Learning: An Overview
• Interleaved Group Convolutions for Deep Neural Networks
• Security Analysis of Deep Neural Networks Operating in the Presence of Cache Side-Channel Attacks
• Deep Convolutional Neural Network Design Patterns
• Light Field Image Quality Assessment With Auxiliary Learning Based on Depthwise and Anglewise Separable Convolutions
• PolyNet: A Pursuit of Structural Diversity in Very Deep Networks
• Deep Convolutional Neural Networks with Merge-and-Run Mappings
• A Novel Weight-Shared Multi-Stage CNN for Scale Robustness
• A Good Practice Towards Top Performance of Face Recognition: Transferred Deep Feature Fusion
• DeepSweep: An Evaluation Framework for Mitigating DNN Backdoor Attacks using Data Augmentation
• Knowledge Transfer for Cross-Domain Reinforcement Learning: A Systematic Review
• Mixing ADAM and SGD: a Combined Optimization Method
• Robust learning with implicit residual networks
• Learning Neuro-symbolic Programs for Language Guided Robot Manipulation
• Color2Embed: Fast Exemplar-Based Image Colorization using Color Embeddings
• Recent Advances in the Applications of Convolutional Neural Networks to Medical Image Contour Detection
• Convolutional Networks with Dense Connectivity
• SWNet: Small-World Neural Networks and Rapid Convergence
• Beyond Forward Shortcuts: Fully Convolutional Master-Slave Networks (MSNets) with Backward Skip Connections for Semantic Segmentation
• Sparsely Aggregated Convolutional Networks
• Action-Based ADHD Diagnosis in Video
• Deep Global-Connected Net With The Generalized Multi-Piecewise ReLU Activation in Deep Learning
• Score-Based Generative Classifiers
• Automated Let's Play Commentary
• Hierarchical Transfer Convolutional Neural Networks for Image Classification
• Automatic Inference of Cross-modal Connection Topologies for X-CNNs
• Distributed U-net model and Image Segmentation for Lung Cancer Detection
• Convergence of backpropagation with momentum for network architectures with skip connections
• Deep Discrete Hashing with Self-supervised Pairwise Labels
• Pseudo-Labeling for Small Lesion Detection on Diabetic Retinopathy Images
• Orthogonal and Idempotent Transformations for Learning Deep Neural Networks
• A Camera free fiber speckle wavemeter
• Compressive Sensing Based Adaptive Defence Against Adversarial Images
• Can a powerful neural network be a teacher for a weaker neural network?
• Multi-scale Convolution Aggregation and Stochastic Feature Reuse for DenseNets
• Research on a New Convolutional Neural Network Model Combined with Random Edges Adding