Box2Poly: Memory-Efficient Polygon Prediction of Arbitrarily Shaped and Rotated Text

Abstract

Recently, Transformer-based text detection techniques have sought to predict polygons by encoding the coordinates of individual boundary vertices using distinct query features. However, this approach incurs a significant memory overhead and struggles to effectively capture the intricate relationships between vertices belonging to the same instance. Consequently, irregular text layouts often lead to the prediction of outlined vertices, diminishing the quality of results. To address these challenges, we present an innovative approach rooted in Sparse R-CNN: a cascade decoding pipeline for polygon prediction. Our method ensures precision by iteratively refining polygon predictions, considering both the scale and location of preceding results. Leveraging this stabilized regression pipeline, even employing just a single feature vector to guide polygon instance regression yields promising detection results. Simultaneously, the leverage of instance-level feature proposal substantially enhances memory efficiency (>50% less vs. the state-of-the-art method DPText-DETR) and reduces inference speed (>40% less vs. DPText-DETR) with minor performance drop on benchmarks.

Accepted to AAAI2024

References in corpus (20)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Deformable DETR: Deformable Transformers for End-to-End Object Detection
• UnitBox: An Advanced Object Detection Network
• Cascade R-CNN: Delving into High Quality Object Detection
• TextField: Learning A Deep Direction Field for Irregular Scene Text Detection
• Learning High-Precision Bounding Box for Rotated Object Detection via Kullback-Leibler Divergence
• Rethinking Rotated Object Detection with Gaussian Wasserstein Distance Loss
• The KFIoU Loss for Rotated Object Detection
• Efficient and Accurate Arbitrary-Shaped Text Detection with Pixel Aggregation Network
• MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction
• Real-time Scene Text Detection with Differentiable Binarization
• ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Network
• ABCNet v2: Adaptive Bezier-Curve Network for Real-time End-to-end Text Spotting
• Look More Than Once: An Accurate Detector for Text of Arbitrary Shapes
• All You Need Is Boundary: Toward Arbitrary-Shaped Text Spotting
• I3CL:Intra- and Inter-Instance Collaborative Learning for Arbitrary-shaped Scene Text Detection
• MOST: A Multi-Oriented Scene Text Detector with Localization Refinement
• DPText-DETR: Towards Better Scene Text Detection with Dynamic Points in Transformer
• DeepSolo: Let Transformer Decoder with Explicit Points Solo for Text Spotting
• Arbitrary Shape Text Detection via Boundary Transformer