Deep High-Resolution Representation Learning for Visual Recognition

Abstract

High-resolution representations are essential for position-sensitive vision problems, such as human pose estimation, semantic segmentation, and object detection. Existing state-of-the-art frameworks first encode the input image as a low-resolution representation through a subnetwork that is formed by connecting high-to-low resolution convolutions \emph{in series} (e.g., ResNet, VGGNet), and then recover the high-resolution representation from the encoded low-resolution representation. Instead, our proposed network, named as High-Resolution Network (HRNet), maintains high-resolution representations through the whole process. There are two key characteristics: (i) Connect the high-to-low resolution convolution streams \emph{in parallel}; (ii) Repeatedly exchange the information across resolutions. The benefit is that the resulting representation is semantically richer and spatially more precise. We show the superiority of the proposed HRNet in a wide range of applications, including human pose estimation, semantic segmentation, and object detection, suggesting that the HRNet is a stronger backbone for computer vision problems. All the codes are available at~{\url{https://github.com/HRNet}}.

To appear in TPAMI. State-of-the-art performance on human pose estimation, semantic segmentation, object detection, instance segmentation, and face alignment. Full version of arXiv:1904.04514. (arXiv admin note: text overlap with arXiv:1904.04514)

References in corpus (28)

• Rethinking Atrous Convolution for Semantic Image Segmentation
• FCOS: Fully Convolutional One-Stage Object Detection
• MMDetection: Open MMLab Detection Toolbox and Benchmark
• High-Resolution Representations for Labeling Pixels and Regions
• Associative Embedding: End-to-End Learning for Joint Detection and Grouping
• Human pose estimation via Convolutional Part Heatmap Regression
• OCNet: Object Context Network for Scene Parsing
• Dual Attention Network for Scene Segmentation
• Dilated Residual Networks
• Hybrid Task Cascade for Instance Segmentation
• Convolutional Neural Fabrics
• CenterNet: Keypoint Triplets for Object Detection
• Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation
• AI Challenger : A Large-scale Dataset for Going Deeper in Image Understanding
• Cascaded Pyramid Network for Multi-Person Pose Estimation
• Scale-Invariant Convolutional Neural Networks
• Cascade R-CNN: High Quality Object Detection and Instance Segmentation
• Stacked Deconvolutional Network for Semantic Segmentation
• Deeply-Fused Nets
• Devil in the Details: Towards Accurate Single and Multiple Human Parsing
• Interlinked Convolutional Neural Networks for Face Parsing
• Interlaced Sparse Self-Attention for Semantic Segmentation
• Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation
• Joint Multi-view Face Alignment in the Wild
• Stacked Dense U-Nets with Dual Transformers for Robust Face Alignment
• Sequential Context Encoding for Duplicate Removal
• M2Det: A Single-Shot Object Detector based on Multi-Level Feature Pyramid Network
• High-Resolution Network for Photorealistic Style Transfer

Cited by in corpus (55)

• Segment Anything Model for Medical Image Analysis: an Experimental Study
• Tent: Fully Test-time Adaptation by Entropy Minimization
• Pixel-in-Pixel Net: Towards Efficient Facial Landmark Detection in the Wild
• Masked-attention Mask Transformer for Universal Image Segmentation
• HRFormer: High-Resolution Transformer for Dense Prediction
• Backbones-Review: Feature Extraction Networks for Deep Learning and Deep Reinforcement Learning Approaches
• FasterSeg: Searching for Faster Real-time Semantic Segmentation
• SuperAnimal pretrained pose estimation models for behavioral analysis
• Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation
• Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision
• LeViT-UNet: Make Faster Encoders with Transformer for Medical Image Segmentation
• Markerless human pose estimation for biomedical applications: a survey
• Prior Guided Feature Enrichment Network for Few-Shot Segmentation
• Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation
• BEV-Seg: Bird's Eye View Semantic Segmentation Using Geometry and Semantic Point Cloud
• SHaRPose: Sparse High-Resolution Representation for Human Pose Estimation
• Simple Training Strategies and Model Scaling for Object Detection
• SpineNet: Learning Scale-Permuted Backbone for Recognition and Localization
• DCNAS: Densely Connected Neural Architecture Search for Semantic Image Segmentation
• Comparison of marker-less 2D image-based methods for infant pose estimation
• Analyzing Worldwide Social Distancing through Large-Scale Computer Vision
• Efficient Differentiable Neural Architecture Search with Meta Kernels
• FaceX-Zoo: A PyTorch Toolbox for Face Recognition
• Boundary-preserving Mask R-CNN
• UTRNet: High-Resolution Urdu Text Recognition In Printed Documents
• WLASL-LEX: a Dataset for Recognising Phonological Properties in American Sign Language
• Proposal Learning for Semi-Supervised Object Detection
• ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training
• Segmenting Transparent Objects in the Wild
• The SpaceNet Multi-Temporal Urban Development Challenge
• PointFlow: Flowing Semantics Through Points for Aerial Image Segmentation
• Structured Convolutions for Efficient Neural Network Design
• DeepPatent2: A Large-Scale Benchmarking Corpus for Technical Drawing Understanding
• hSDB-instrument: Instrument Localization Database for Laparoscopic and Robotic Surgeries
• PFENet++: Boosting Few-shot Semantic Segmentation with the Noise-filtered Context-aware Prior Mask
• 1st Place Solutions of Waymo Open Dataset Challenge 2020 -- 2D Object Detection Track
• Identifying Crucial Objects in Blind and Low-Vision Individuals' Navigation
• Dilated SpineNet for Semantic Segmentation
• Unsupervised 3D Learning for Shape Analysis via Multiresolution Instance Discrimination
• Densely connected multidilated convolutional networks for dense prediction tasks
• A Self-Supervised Feature Map Augmentation (FMA) Loss and Combined Augmentations Finetuning to Efficiently Improve the Robustness of CNNs
• Combining detection and tracking for human pose estimation in videos
• Learning Effective Visual Relationship Detector on 1 GPU
• FNA++: Fast Network Adaptation via Parameter Remapping and Architecture Search
• EfficientHRNet: Efficient Scaling for Lightweight High-Resolution Multi-Person Pose Estimation
• Semantic Segmentation of Radar Detections using Convolutions on Point Clouds
• Search to Distill: Pearls are Everywhere but not the Eyes
• Self-Parameterization Based Multi-Resolution Mesh Convolution Networks
• Context Prior for Scene Segmentation
• Learning to Segment Anatomical Structures Accurately from One Exemplar
• Dense-Resolution Network for Point Cloud Classification and Segmentation
• Semantic Segmentation on VSPW Dataset through Aggregation of Transformer Models
• Separating Content and Style for Unsupervised Image-to-Image Translation
• BiCANet: Bi-directional Contextual Aggregating Network for Image Semantic Segmentation
• Affect-DML: Context-Aware One-Shot Recognition of Human Affect using Deep Metric Learning