10 papers
DeLoop: Repeated Actions as Self-Supervision for Long-Horizon Agents
Tianle Chen, Pushkar Singh, Ketan Todi +8
Language-model agents that run long tasks often redo work they have already done: they re-issue a tool call they made earlier and get back exactly the same observation, a step that…
Swift Sampling: Selecting Temporal Surprises via Taylor Series
Dahye Kim, Bhuvan Sachdeva, Karan Uppal +3
While most frames in long-form video are redundant, the critical information resides in temporal surprises: moments where the actual visual features deviate from their predicted ev…
FAGER: Factually Grounded Evaluation and Refinement of Text-to-Image Models
Youngsun Lim, Cusuh Ham, Pin-Yu Chen +1
Existing text-to-image (T2I) evaluation metrics mainly assess whether generated images align with information explicitly stated in the prompt, but often fail to capture factual req…
FuTCR: Future-Targeted Contrast and Repulsion for Continual Panoptic Segmentation
Nicholas Ikechukwu, Keanu Nichols, Deepti Ghadiyaram +1
Continual Panoptic Segmentation (CPS) requires methods that can quickly adapt to new categories over time. The nature of this dense prediction task means that training images may c…
A Systematic Study of Cross-Modal Typographic Attacks on Audio-Visual Reasoning
Tianle Chen, Deepti Ghadiyaram
As audio-visual multi-modal large language models (MLLMs) are increasingly deployed in safety-critical applications, understanding their cross-modal vulnerabilities is essential. T…
Semantic Richness or Geometric Reasoning? The Fragility of VLM's Visual Invariance
Jason Qiu, Zachary Meurer, Xavier Thomas +1
This work investigates the fundamental fragility of state-of-the-art Vision-Language Models (VLMs) under basic geometric transformations. While modern VLMs excel at semantic tasks…
Seeing Isn't Orienting: A Cognitively Grounded Benchmark Reveals Systematic Orientation Failures in MLLMs Supplementary
Nazia Tasnim, Keanu Nichols, Yuting Yang +4
Humans learn object orientation progressively, from recognizing which way an object faces, to mentally rotating it, to reasoning about orientations between objects. Current vision-…
DDiT: Dynamic Patch Scheduling for Efficient Diffusion Transformers
Dahye Kim, Deepti Ghadiyaram, Raghudeep Gadde
Diffusion Transformers (DiTs) have achieved state-of-the-art performance in image and video generation, but their success comes at the cost of heavy computation. This inefficiency…
Seeing Isn't Orienting: A Cognitively Informed Hierarchical Benchmark for Object Orientation in MLLMs
Nazia Tasnim, Keanu Nichols, Yuting Yan +4
Humans develop object orientation understanding progressively, from recognizing which way an object faces to reasoning about orientations across multiple objects. Yet existing visi…
Concept Steerers: Leveraging K-Sparse Autoencoders for Test-Time Controllable Generations
Dahye Kim, Deepti Ghadiyaram
Despite the remarkable progress in text-to-image generative models, they are prone to adversarial attacks and inadvertently generate unsafe, unethical content. Existing approaches…