High-Resolution Representations for Labeling Pixels and Regions

Abstract

High-resolution representation learning plays an essential role in many vision problems, e.g., pose estimation and semantic segmentation. The high-resolution network (HRNet)~\cite{SunXLW19}, recently developed for human pose estimation, maintains high-resolution representations through the whole process by connecting high-to-low resolution convolutions in \emph{parallel} and produces strong high-resolution representations by repeatedly conducting fusions across parallel convolutions. In this paper, we conduct a further study on high-resolution representations by introducing a simple yet effective modification and apply it to a wide range of vision tasks. We augment the high-resolution representation by aggregating the (upsampled) representations from all the parallel convolutions rather than only the representation from the high-resolution convolution as done in~\cite{SunXLW19}. This simple modification leads to stronger representations, evidenced by superior results. We show top results in semantic segmentation on Cityscapes, LIP, and PASCAL Context, and facial landmark detection on AFLW, COFW, $300$W, and WFLW. In addition, we build a multi-level representation from the high-resolution representation and apply it to the Faster R-CNN object detection framework and the extended frameworks. The proposed approach achieves superior results to existing single-model networks on COCO object detection. The code and models have been publicly available at \url{https://github.com/HRNet}.

References in corpus (10)

• Rethinking Atrous Convolution for Semantic Image Segmentation
• Human pose estimation via Convolutional Part Heatmap Regression
• Dilated Residual Networks
• Rethinking ImageNet Pre-training
• Devil in the Details: Towards Accurate Single and Multiple Human Parsing
• Interlinked Convolutional Neural Networks for Face Parsing
• Look into Person: Self-supervised Structure-sensitive Learning and A New Benchmark for Human Parsing
• Joint Multi-view Face Alignment in the Wild
• Stacked Dense U-Nets with Dual Transformers for Robust Face Alignment
• M2Det: A Single-Shot Object Detector based on Multi-Level Feature Pyramid Network

Cited by in corpus (96)

• Res2Net: A New Multi-scale Backbone Architecture
• MMDetection: Open MMLab Detection Toolbox and Benchmark
• OCNet: Object Context Network for Scene Parsing
• Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation
• Deep High-Resolution Representation Learning for Visual Recognition
• Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey
• Satellite Pose Estimation Challenge: Dataset, Competition Design and Results
• Cross-view Semantic Segmentation for Sensing Surroundings
• MAP-Net: Multi Attending Path Neural Network for Building Footprint Extraction from Remote Sensed Imagery
• Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes
• Deep Learning for Generic Object Detection: A Survey
• An Empirical Study of Remote Sensing Pretraining
• Using DUCK-Net for Polyp Image Segmentation
• HRTransNet: HRFormer-Driven Two-Modality Salient Object Detection
• A Survey on Deep Learning-Based Monocular Spacecraft Pose Estimation: Current State, Limitations and Prospects
• Precise Facial Landmark Detection by Reference Heatmap Transformer
• UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation
• PhenoBench -- A Large Dataset and Benchmarks for Semantic Image Interpretation in the Agricultural Domain
• KPRNet: Improving projection-based LiDAR semantic segmentation
• On the Real-World Adversarial Robustness of Real-Time Semantic Segmentation Models for Autonomous Driving
• CitySurfaces: City-Scale Semantic Segmentation of Sidewalk Materials
• FFR_FD: Effective and Fast Detection of DeepFakes Based on Feature Point Defects
• Learning Texture Transformer Network for Image Super-Resolution
• TNCR: Table Net Detection and Classification Dataset
• Deep Weakly-Supervised Learning Methods for Classification and Localization in Histology Images: A Survey
• Attention guided global enhancement and local refinement network for semantic segmentation
• Improving Semantic Segmentation via Self-Training
• A Survey on Masked Facial Detection Methods and Datasets for Fighting Against COVID-19
• Learning Independent Instance Maps for Crowd Localization
• Scaling Wide Residual Networks for Panoptic Segmentation
• Boosting RGB-D Saliency Detection by Leveraging Unlabeled RGB Images
• LOTR: Face Landmark Localization Using Localization Transformer
• Learning to Downsample for Segmentation of Ultra-High Resolution Images
• Face X-ray for More General Face Forgery Detection
• Multimodal Detection of Unknown Objects on Roads for Autonomous Driving
• Hformer: Hybrid CNN-Transformer for Fringe Order Prediction in Phase Unwrapping of Fringe Projection
• The Federated Tumor Segmentation (FeTS) Challenge
• Shape Controllable Virtual Try-on for Underwear Models
• Clustered Object Detection in Aerial Images
• A Review on Human Pose Estimation
• SegFix: Model-Agnostic Boundary Refinement for Segmentation
• See More Than Once -- Kernel-Sharing Atrous Convolution for Semantic Segmentation
• MaskFace: multi-task face and landmark detector
• LUVLi Face Alignment: Estimating Landmarks' Location, Uncertainty, and Visibility Likelihood
• Learning Dynamic Routing for Semantic Segmentation
• Gaze Estimation with an Ensemble of Four Architectures
• Improving the Efficiency and Robustness of Deepfakes Detection through Precise Geometric Features
• Disentangled Non-Local Neural Networks
• HIH: Towards More Accurate Face Alignment via Heatmap in Heatmap
• MSCFNet: A Lightweight Network With Multi-Scale Context Fusion for Real-Time Semantic Segmentation
• Efficient Semantic Video Segmentation with Per-frame Inference
• HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping
• Decision-based Universal Adversarial Attack
• S3Net: Innovating Stereo Matching and Semantic Segmentation with a Single-Branch Semantic Stereo Network in Satellite Epipolar Imagery
• Feature Pyramid Transformer
• Robust Semantic Segmentation with Superpixel-Mix
• Efficient onboard multi-task AI architecture based on self-supervised learning
• Combining detection and tracking for human pose estimation in videos
• CaseNet: Content-Adaptive Scale Interaction Networks for Scene Parsing
• Whole-Body Human Pose Estimation in the Wild
• RiWNet: A moving object instance segmentation Network being Robust in adverse Weather conditions
• Monocular Real-Time Volumetric Performance Capture
• FlatteNet: A Simple Versatile Framework for Dense Pixelwise Prediction
• Canonical 3D Deformer Maps: Unifying parametric and non-parametric methods for dense weakly-supervised category reconstruction
• EfficientHRNet: Efficient Scaling for Lightweight High-Resolution Multi-Person Pose Estimation
• Instance-Level Task Parameters: A Robust Multi-task Weighting Framework
• FarSee-Net: Real-Time Semantic Segmentation by Efficient Multi-scale Context Aggregation and Feature Space Super-resolution
• Directional Temporal Modeling for Action Recognition
• Deep Consensus Learning
• Spirit Distillation: Precise Real-time Semantic Segmentation of Road Scenes with Insufficient Data
• Leveraging Structured Pruning of Convolutional Neural Networks
• HR-RCNN: Hierarchical Relational Reasoning for Object Detection
• Fast and Accurate: Structure Coherence Component for Face Alignment
• Foveation for Segmentation of Ultra-High Resolution Images
• End-to-end Deep Learning Methods for Automated Damage Detection in Extreme Events at Various Scales
• 2D Wasserstein Loss for Robust Facial Landmark Detection
• Gaussian Vector: An Efficient Solution for Facial Landmark Detection
• ICDAR 2021 Competition on Components Segmentation Task of Document Photos
• Discover the Mysteries of the Maya: Selected Contributions from the Machine Learning Challenge & The Discovery Challenge Workshop at ECML PKDD 2021
• Energy Consumption Analysis of pruned Semantic Segmentation Networks on an Embedded GPU
• Contrastive Learning and Self-Training for Unsupervised Domain Adaptation in Semantic Segmentation
• Location-aware Upsampling for Semantic Segmentation
• Self-Learning with Rectification Strategy for Human Parsing
• Attribute-guided Feature Extraction and Augmentation Robust Learning for Vehicle Re-identification
• TraceCaps: A Capsule-based Neural Network for Semantic Segmentation
• Weight mechanism: adding a constant in concatenation of series connect
• KCP: Kernel Cluster Pruning for Dense Labeling Neural Networks
• Dense Multiscale Feature Fusion Pyramid Networks for Object Detection in UAV-Captured Images
• PDFNet: Pointwise Dense Flow Network for Urban-Scene Segmentation
• Affinity-aware Compression and Expansion Network for Human Parsing
• Think about boundary: Fusing multi-level boundary information for landmark heatmap regression
• Accurate Fine-grained Layout Analysis for the Historical Tibetan Document Based on the Instance Segmentation
• Artificial Intelligence in Surgery: Neural Networks and Deep Learning
• 3rd Place Solution for Short-video Face Parsing Challenge
• VisDrone-CC2020: The Vision Meets Drone Crowd Counting Challenge Results
• GSTO: Gated Scale-Transfer Operation for Multi-Scale Feature Learning in Pixel Labeling