5 citations · 5 across the 11 of their papers we have counts for
11 papers
START: Spatial and Textual Learning for Chart Understanding
Zhuoming Liu, Xiaofeng Gao, Feiyang Niu +3
Chart understanding is crucial for deploying multimodal large language models (MLLMs) in real-world scenarios such as analyzing scientific papers and technical reports. Unlike natu…
VaPR -- Vision-language Preference alignment for Reasoning
Rohan Wadhawan, Fabrice Y Harel-Canada, Zi-Yi Dou +3
Preference finetuning methods like Direct Preference Optimization (DPO) with AI-generated feedback have shown promise in aligning Large Vision-Language Models (LVLMs) with human pr…
MDSEval: A Meta-Evaluation Benchmark for Multimodal Dialogue Summarization
Yinhong Liu, Jianfeng He, Hang Su +6
Multimodal Dialogue Summarization (MDS) is a critical task with wide-ranging applications. To support the development of effective MDS models, robust automatic evaluation methods a…
TemMed-Bench: Evaluating Temporal Medical Image Reasoning in Vision-Language Models
Junyi Zhang, Jia-Chen Gu, Wenbo Hu +3
Existing medical reasoning benchmarks for vision-language models primarily focus on analyzing a patient's condition based on an image from a single visit. However, this setting dev…
InSTA: Towards Internet-Scale Training For Agents
Brandon Trabucco, Gunnar Sigurdsson, Robinson Piramuthu +1
The predominant approach for training web navigation agents is to gather human demonstrations for a set of popular websites and hand-written tasks, but it is becoming clear that hu…
T2V-Turbo-v2: Enhancing Video Generation Model Post-Training through Data, Reward, and Conditional Guidance Design
Jiachen Li, Qian Long, Jian Zheng +4
In this paper, we focus on enhancing a diffusion-based text-to-video (T2V) model during the post-training phase by distilling a highly capable consistency model from a pretrained T…
FlexEControl: Flexible and Efficient Multimodal Control for Text-to-Image Generation
Xuehai He, Jian Zheng, Jacob Zhiyuan Fang +6
Controllable text-to-image (T2I) diffusion models generate images conditioned on both text prompts and semantic inputs of other modalities like edge maps. Nevertheless, current con…
Is the House Ready For Sleeptime? Generating and Evaluating Situational Queries for Embodied Question Answering
Vishnu Sashank Dorbala, Prasoon Goyal, Robinson Piramuthu +3
We present and tackle the problem of Embodied Question Answering (EQA) with Situational Queries (S-EQA) in a household environment. Unlike prior EQA work tackling simple queries th…
"Don't forget to put the milk back!" Dataset for Enabling Embodied Agents to Detect Anomalous Situations
James F. Mullen, Prasoon Goyal, Robinson Piramuthu +3
Home robots intend to make their users lives easier. Our work assists in this goal by enabling robots to inform their users of dangerous or unsanitary anomalies in their home. Some…
E-ViLM: Efficient Video-Language Model via Masked Video Modeling with Semantic Vector-Quantized Tokenizer
Jacob Zhiyuan Fang, Skyler Zheng, Vasu Sharma +1
To build scalable models for challenging real-world tasks, it is important to learn from diverse, multi-modal data in various forms (e.g., videos, text, and images). Among the exis…
Characterizing Video Question Answering with Sparsified Inputs
Shiyuan Huang, Robinson Piramuthu, Vicente Ordonez +2
In Video Question Answering, videos are often processed as a full-length sequence of frames to ensure minimal loss of information. Recent works have demonstrated evidence that spar…