1 citations · 4 across the 10 of their papers we have counts for
10 papers
Towards GUI Agents: Vision-Language Diffusion Models for GUI Grounding
Shrinidhi Kumbhar, Haofu Liao, Srikar Appalaraju +1
Autoregressive (AR) vision-language models (VLMs) have long dominated multimodal understanding, reasoning, and graphical user interface (GUI) grounding. Recently, discrete diffusio…
Turbocharging Web Automation: The Impact of Compressed History States
Xiyue Zhu, Peng Tang, Haofu Liao +1
Language models have led to a leap forward in web automation. The current web automation approaches take the current web state, history actions, and language instruction as inputs…
R-VLM: Region-Aware Vision Language Model for Precise GUI Grounding
Joonhyung Park, Peng Tang, Sagnik Das +4
Visual agent models for automating human activities on Graphical User Interfaces (GUIs) have emerged as a promising research direction, driven by advances in large Vision Language…
DocKD: Knowledge Distillation from LLMs for Open-World Document Understanding Models
Sungnyun Kim, Haofu Liao, Srikar Appalaraju +6
Visual document understanding (VDU) is a challenging task that involves understanding documents across various modalities (text and image) and layouts (forms, tables, etc.). This s…
VisFocus: Prompt-Guided Vision Encoders for OCR-Free Dense Document Understanding
Ofir Abramovich, Niv Nayman, Sharon Fogel +7
In recent years, notable advancements have been made in the domain of visual document understanding, with the prevailing architecture comprising a cascade of vision and language mo…
RAVEN: Multitask Retrieval Augmented Vision-Language Learning
Varun Nagaraj Rao, Siddharth Choudhary, Aditya Deshpande +2
The scaling of large language models to encode all the world's knowledge in model parameters is unsustainable and has exacerbated resource barriers. Retrieval-Augmented Generation…
Enhancing Vision-Language Pre-training with Rich Supervisions
Yuan Gao, Kunyu Shi, Pengkai Zhu +7
We propose Strongly Supervised pre-training with ScreenShots (S4) - a novel pre-training paradigm for Vision-Language Models using data from large-scale web screenshot rendering. U…
DEED: Dynamic Early Exit on Decoder for Accelerating Encoder-Decoder Transformer Models
Peng Tang, Pengkai Zhu, Tian Li +3
Encoder-decoder transformer models have achieved great success on various vision-language (VL) tasks, but they suffer from high inference latency. Typically, the decoder takes up m…
Multiple-Question Multiple-Answer Text-VQA
Peng Tang, Srikar Appalaraju, R. Manmatha +2
We present Multiple-Question Multiple-Answer (MQMA), a novel approach to do text-VQA in encoder-decoder transformer models. The text-VQA task requires a model to answer a question…
A Multi-Modal Multilingual Benchmark for Document Image Classification
Yoshinari Fujinuma, Siddharth Varia, Nishant Sankaran +3
Document image classification is different from plain-text document classification and consists of classifying a document by understanding the content and structure of documents su…