4 citations · 7 across the 18 of their papers we have counts for
19 papers
Long-Horizon Embodied Decision-Making via Multimodal Memory Compression
Bingxuan Li, Rui Yang, Cheng Qian +6
Agents are increasingly expected to act not only as task executors, but also as decision-makers on behalf of human users. This shift requires agents to accumulate evidence over lon…
Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment
Dwip Dalal, Shivansh Patel, Chahit Jain +8
Finetuning a pretrained vision-language model (VLM) on robot demonstrations via behavior cloning (BC) has become the standard recipe for vision-language-action (VLA) policies. Howe…
AdaPlanBench: Evaluating Adaptive Planning in Large Language Model Agents under World and User Constraints
Jiayu Liu, Cheng Qian, Zhenhailong Wang +10
Planning for real-world problems by language models often involves both world and user constraints, which may not be fully specified upfront and are progressively disclosed through…
Advancing Creative Physical Intelligence in Large Multimodal Models
Cheng Qian, Hyeonjeong Ha, Jiayu Liu +10
Large multimodal models (LMMs) have rapidly advanced in perception and reasoning; however, it remains unclear whether these capabilities generalize to discovering visually grounded…
MemGuard: Preventing Memory Contamination in Long-Term Memory-Augmented Large Language Models
Hyeonjeong Ha, Jeonghwan Kim, Cheng Qian +7
Memory-augmented large language models extend reasoning beyond a fixed context window by maintaining long-term memory across interactions. However, existing memory systems often co…
OSExpert: Computer-Use Agents Learning Professional Skills via Exploration
Jiateng Liu, Zhenhailong Wang, Rushi Wang +6
General-purpose computer-use agents have shown impressive performance across diverse digital environments. However, our new benchmark, OSExpert-Eval, indicates they remain far less…
PixSearch: Region-Grounded Retrieval for Knowledgeable Large Multimodal Models
Jeonghwan Kim, Renjie Tao, Sanat Sharma +8
Multimodal Retrieval-Augmented Generation (MM-RAG) augments visual question answering (VQA) with external knowledge, but many systems rely on modular pipelines that separate visual…
PEARL: Self-Evolving Assistant for Time Management with Reinforcement Learning
Bingxuan Li, Jeonghwan Kim, Cheng Qian +4
Overlapping calendar invitations force busy professionals to repeatedly decide which meetings to attend, reschedule, or decline. We refer to this preference-driven decision process…
Constructive Distortion: Improving MLLMs with Attention-Guided Image Warping
Dwip Dalal, Gautam Vashishtha, Utkarsh Mishra +6
Multimodal large language models (MLLMs) often miss small details and spatial relations in cluttered scenes, leading to errors in fine-grained perceptual grounding. We introduce At…
PARTONOMY: Large Multimodal Models with Part-Level Visual Understanding
Ansel Blume, Jeonghwan Kim, Hyeonjeong Ha +7
Real-world objects are composed of distinctive, object-specific parts. Identifying these parts is key to performing fine-grained, compositional reasoning-yet, large multimodal mode…
MM-PoisonRAG: Disrupting Multimodal RAG with Local and Global Poisoning Attacks
Hyeonjeong Ha, Qiusi Zhan, Jeonghwan Kim +6
Retrieval-augmented generation (RAG) has become a common practice in multimodal large language models (MLLM) to enhance factual grounding and reduce hallucination. Yet, its relianc…
SYNTHIA: Novel Concept Design with Affordance Composition
Hyeonjeong Ha, Xiaomeng Jin, Jeonghwan Kim +7
Text-to-image (T2I) models enable rapid concept design, making them widely used in AI-driven design. While recent studies focus on generating semantic and stylistic variations of g…
Infogent: An Agent-Based Framework for Web Information Aggregation
Revanth Gangi Reddy, Sagnik Mukherjee, Jeonghwan Kim +3
Despite seemingly performant web agents on the task-completion benchmarks, most existing methods evaluate the agents based on a presupposition: the web navigation task consists of…
Aligning LLMs with Individual Preferences via Interaction
Shujin Wu, May Fung, Cheng Qian +3
As large language models (LLMs) demonstrate increasingly advanced capabilities, aligning their behaviors with human values and preferences becomes crucial for their wide adoption.…
ARMADA: Attribute-Based Multimodal Data Augmentation
Xiaomeng Jin, Jeonghwan Kim, Yu Zhou +4
In Multimodal Language Models (MLMs), the cost of manually annotating high-quality image-text pair data for fine-tuning and alignment is extremely high. While existing multimodal d…
Finer: Investigating and Enhancing Fine-Grained Visual Concept Recognition in Large Vision Language Models
Jeonghwan Kim, Heng Ji
Recent advances in instruction-tuned Large Vision-Language Models (LVLMs) have imbued the models with the ability to generate high-level, image-grounded explanations with ease. Whi…
Why So Gullible? Enhancing the Robustness of Retrieval-Augmented Models against Counterfactual Noise
Giwon Hong, Jeonghwan Kim, Junmo Kang +2
Most existing retrieval-augmented language models (LMs) assume a naive dichotomy within a retrieved document set: query-relevance and irrelevance. Our work investigates a more chal…
Maximizing Efficiency of Language Model Pre-training for Learning Representation
Junmo Kang, Suwon Shin, Jeonghwan Kim +2
Pre-trained language models in the past years have shown exponential growth in model parameters and compute time. ELECTRA is a novel approach for improving the compute efficiency o…
Leveraging Order-Free Tag Relations for Context-Aware Recommendation
Junmo Kang, Jeonghwan Kim, Suwon Shin +1
Tag recommendation relies on either a ranking function for top- tags or an autoregressive generation method. However, the previous methods neglect one of two seemingly conflicti…