7 papers
Zoom to Essence: Trainless GUI Grounding by Inferring upon Interface Elements
Ziwei Liu, Tao Feng, Borui Kang +2
Multimodal Large Language Model (MLLM)-based Graphical User Interface (GUI) agents develop rapidly, with visual grounding that maps natural language instructions to target UI eleme…
SceneAssistant: A Visual Feedback Agent for Open-Vocabulary 3D Scene Generation
Jun Luo, Jiaxiang Tang, Ruijie Lu +1
Text-to-3D scene generation from natural language is highly desirable for digital content creation. However, existing methods are largely domain-restricted or reliant on predefined…
TDR: Task-Decoupled Retrieval with Fine-Grained LLM Feedback for In-Context Learning
Yifu Chen, Bingchen Huang, Zhiling Wang +4
In-context learning (ICL) has become a classic approach for enabling LLMs to handle various tasks based on a few input-output examples. The effectiveness of ICL heavily relies on t…
Marten: Visual Question Answering with Mask Generation for Multi-modal Document Understanding
Zining Wang, Tongkun Guan, Pei Fu +7
Multi-modal Large Language Models (MLLMs) have introduced a novel dimension to document understanding, i.e., they endow large language models with visual comprehension capabilities…
A Token-level Text Image Foundation Model for Document Understanding
Tongkun Guan, Zining Wang, Pei Fu +9
In recent years, general visual foundation models (VFMs) have witnessed increasing adoption, particularly as image encoders for popular multi-modal large language models (MLLMs). H…
Multimodal Large Language Models for Text-rich Image Understanding: A Comprehensive Review
Pei Fu, Tongkun Guan, Zining Wang +8
The recent emergence of Multi-modal Large Language Models (MLLMs) has introduced a new dimension to the Text-rich Image Understanding (TIU) field, with models demonstrating impress…