125 citations · 166 across the 36 of their papers we have counts for
20 papers · 1 filter
CC-OCR: A Comprehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in Literacy
Zhibo Yang, Jun Tang, Zhaohai Li +9
Large Multimodal Models (LMMs) have demonstrated impressive performance in recognizing document images with natural language instructions. However, it remains unclear to what exten…
Liquid: Language Models are Scalable and Unified Multi-modal Generators
Junfeng Wu, Yi Jiang, Chuofan Ma +5
We present Liquid, an auto-regressive generation paradigm that seamlessly integrates visual comprehension and generation by tokenizing images into discrete codes and learning these…
Theorem-Validated Reverse Chain-of-Thought Problem Generation for Geometric Reasoning
Linger Deng, Linghao Zhu, Yuliang Liu +6
Large Multimodal Models (LMMs) face limitations in geometric reasoning due to insufficient Chain of Thought (CoT) image-text training data. While existing approaches leverage templ…
Mini-Monkey: Alleviating the Semantic Sawtooth Effect for Lightweight MLLMs via Complementary Image Pyramid
Mingxin Huang, Yuliang Liu, Dingkang Liang +2
Recently, scaling images to high resolution has received much attention in multimodal large language models (MLLMs). Most existing practices adopt a sliding-window-style cropping s…
MoE Jetpack: From Dense Checkpoints to Adaptive Mixture of Experts for Vision Tasks
Xingkui Zhu, Yiran Guan, Dingkang Liang +3
The sparsely activated mixture of experts (MoE) model presents a promising alternative to traditional densely activated (dense) models, enhancing both quality and computational eff…
Puzzle Pieces Picker: Deciphering Ancient Chinese Characters with Radical Reconstruction
Pengjie Wang, Kaile Zhang, Xinyu Wang +5
Oracle Bone Inscriptions is one of the oldest existing forms of writing in the world. However, due to the great antiquity of the era, a large number of Oracle Bone Inscriptions (OB…
Deciphering Oracle Bone Language with Diffusion Models
Haisu Guan, Huanxin Yang, Xinyu Wang +5
Originating from China's Shang Dynasty approximately 3,000 years ago, the Oracle Bone Script (OBS) is a cornerstone in the annals of linguistic history, predating many established…
Dataset and Benchmark for Urdu Natural Scenes Text Detection, Recognition and Visual Question Answering
Hiba Maryam, Ling Fu, Jiajun Song +4
The development of Urdu scene text detection, recognition, and Visual Question Answering (VQA) technologies is crucial for advancing accessibility, information retrieval, and lingu…
The First Swahili Language Scene Text Detection and Recognition Dataset
Fadila Wendigoundi Douamba, Jianjun Song, Ling Fu +2
Scene text recognition is essential in many applications, including automated translation, information retrieval, driving assistance, and enhancing accessibility for individuals wi…
Exploring the Capabilities of Large Multimodal Models on Dense Text
Shuo Zhang, Biao Yang, Zhang Li +3
While large multi-modal models (LMM) have shown notable progress in multi-modal tasks, their capabilities in tasks involving dense textual content remains to be fully explored. Den…
MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answering
Jingqun Tang, Qi Liu, Yongjie Ye +14
Text-Centric Visual Question Answering (TEC-VQA) in its proper format not only facilitates human-machine interaction in text-centric visual environments but also serves as a de fac…
Bridging the Gap Between End-to-End and Two-Step Text Spotting
Mingxin Huang, Hongliang Li, Yuliang Liu +2
Modularity plays a crucial role in the development and maintenance of complex systems. While end-to-end text spotting efficiently mitigates the issues of error accumulation and sub…
VimTS: A Unified Video and Image Text Spotter for Enhancing the Cross-domain Generalization
Yuliang Liu, Mingxin Huang, Hao Yan +6
Text spotting, a task involving the extraction of textual information from image or video sequences, faces challenges in cross-domain adaption, such as image-to-image and image-to-…
TextSquare: Scaling up Text-Centric Visual Instruction Tuning
Jingqun Tang, Chunhui Lin, Zhen Zhao +15
Text-centric visual question answering (VQA) has made great strides with the development of Multimodal Large Language Models (MLLMs), yet open-source models still fall short of lea…
OmniParser: A Unified Framework for Text Spotting, Key Information Extraction and Table Recognition
Jianqiang Wan, Sibo Song, Wenwen Yu +6
Recently, visually-situated text parsing (VsTP) has experienced notable advancements, driven by the increasing demand for automated document understanding and the emergence of Gene…
TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document
Yuliang Liu, Biao Yang, Qiang Liu +4
We present TextMonkey, a large multimodal model (LMM) tailored for text-centric tasks. Our approach introduces enhancement across several dimensions: By adopting Shifted Window Att…
An open dataset for the evolution of oracle bone characters: EVOBC
Haisu Guan, Jinpeng Wan, Yuliang Liu +6
The earliest extant Chinese characters originate from oracle bone inscriptions, which are closely related to other East Asian languages. These inscriptions hold immense value for a…
Toward Real Text Manipulation Detection: New Dataset and New Solution
Dongliang Luo, Yuliang Liu, Rui Yang +4
With the surge in realistic text tampering, detecting fraudulent text in images has gained prominence for maintaining information security. However, the high costs associated with…
An open dataset for oracle bone script recognition and decipherment
Pengjie Wang, Kaile Zhang, Xinyu Wang +8
Oracle bone script, one of the earliest known forms of ancient Chinese writing, presents invaluable research materials for scholars studying the humanities and geography of the Sha…
SwinTextSpotter v2: Towards Better Synergy for Scene Text Spotting
Mingxin Huang, Dezhi Peng, Hongliang Li +6
End-to-end scene text spotting, which aims to read the text in natural images, has garnered significant attention in recent years. However, recent state-of-the-art methods usually…