2.4k citations · 2.9k across the 65 of their papers we have counts for
71 papers
Echoverse: Deep, Evolving Environments for Training Computer-Use Agents at Scale
Yash Pandya, Sahil Gupta, Sarthak Harne +10
Computer-use agents learn from what their actions change, so training one needs applications it can act on, break and reset. The applications that matter most are login-gated and s…
Fara-1.5: Scalable Learning Environments for Computer Use Agents
Ahmed Awadallah, Sahil Gupta, Yash Lara +12
Collecting computer use data from human demonstrations is expensive and slow, motivating the need for scalable generation strategies. This requires two key ingredients: environment…
ReVision: Scaling Computer-Use Agents via Temporal Visual Redundancy Reduction
Amirhossein Abaskohi, Yuhang He, Peter West +3
Computer-use agents (CUAs) rely on visual observations of graphical user interfaces, where each screenshot is encoded into a large number of visual tokens. As interaction trajector…
Where Did It Go Wrong? Process-Level Evaluation of Web Agents with Semantic State Tracking
Jiwan Chung, JiHyuk Byun, Vibhav Vineet +1
Web agents act through long interaction sequences, yet existing benchmarks evaluate only terminal success, discarding all process information and offering little guidance on improv…
StreamReady: Learning What to Answer and When in Long Streaming Videos
Shehreen Azad, Vibhav Vineet, Yogesh Singh Rawat
Streaming video understanding often involves time-sensitive scenarios where models need to answer exactly when the supporting visual evidence appears: answering before the evidence…
OS-Marathon: Benchmarking Computer-Use Agents on Vast-Horizon, Repetitive Tasks
Jing Wu, Wenjie Ai, Daphne Barretto +7
Vast-horizon, repetitive workflows are common in daily routines, e.g., processing expense reports from a stack of receipts, organising a collection of PDF annotations into structur…
CoSPlan: Corrective Sequential Planning via Scene Graph Incremental Updates
Shresth Grover, Priyank Pathak, Akash Kumar +1
Vision Language Models (VLMs) have shown promising planning capabilities, yet their success remains confined to the text domain, leaving visual decision-making relatively underexpl…
Fara-7B: An Efficient Agentic Model for Computer Use
Ahmed Awadallah, Yash Lara, Raghav Magazine +9
Progress in computer use agents (CUAs) has been constrained by the absence of large and high-quality datasets that capture how humans interact with a computer. While LLMs have thri…
Physics Knowledge in Frontier Models: A Diagnostic Study of Failure Modes
Ieva Bagdonaviciute, Vibhav Vineet
While recent Vision-Language Models (VLMs) have achieved impressive progress, it remains difficult to determine why they succeed or fail on complex reasoning tasks. Traditional ben…
Just Do It!? Computer-Use Agents Exhibit Blind Goal-Directedness
Erfan Shayegani, Keegan Hines, Yue Dong +6
Computer-Use Agents (CUAs) are an increasingly deployed class of agents that take actions on GUIs to accomplish user goals. In this paper, we show that CUAs consistently exhibit Bl…
What MLLMs Learn about When they Learn about Multimodal Reasoning
Jiwan Chung, Neel Joshi, Pratyusha Sharma +2
Evaluation of multimodal reasoning models is typically reduced to a single accuracy score, implicitly treating reasoning as a unitary capability. We introduce MathLens, a benchmark…
OmViD: Omni-supervised active learning for video action detection
Aayush Rana, Akash Kumar, Vibhav Vineet +1
Video action detection requires dense spatio-temporal annotations, which are both challenging and expensive to obtain. However, real-world videos often vary in difficulty and may n…
Test-time Prompt Refinement for Text-to-Image Models
Mohammad Abdul Hafeez Khan, Yash Jain, Siddhartha Bhattacharyya +1
Text-to-image (T2I) generation models have made significant strides but still struggle with prompt sensitivity: even minor changes in prompt wording can yield inconsistent or inacc…
Out of Sight, Not Out of Context? Egocentric Spatial Reasoning in VLMs Across Disjoint Frames
Sahithya Ravi, Gabriel Sarch, Vibhav Vineet +2
An embodied AI assistant operating on egocentric video must integrate spatial cues across time - for instance, determining where an object A, glimpsed a few moments ago lies relati…
TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action
Jen-Hao Cheng, Vivian Wang, Huayu Wang +11
Understanding causal event relationships and achieving fine-grained temporal grounding in videos remain challenging for vision-language models (VLMs). We propose TEMPURA (Temporal…
Grounding Task Assistance with Multimodal Cues from a Single Demonstration
Gabriel Sarch, Balasaravanan Thoravi Kumaravel, Sahithya Ravi +2
A person's demonstration often serves as a key reference for others learning the same task. However, RGB video, the dominant medium for representing these demonstrations, often fai…
Phi-4-reasoning Technical Report
Marah Abdin, Sahaj Agarwal, Ahmed Awadallah +20
We introduce Phi-4-reasoning, a 14-billion parameter reasoning model that achieves strong performance on complex reasoning tasks. Trained via supervised fine-tuning of Phi-4 on car…
HierarQ: Task-Aware Hierarchical Q-Former for Enhanced Video Understanding
Shehreen Azad, Vibhav Vineet, Yogesh Singh Rawat
Despite advancements in multimodal large language models (MLLMs), current approaches struggle in medium-to-long video understanding due to frame and context length limitations. As…
A Large-Scale Analysis on Contextual Self-Supervised Video Representation Learning
Akash Kumar, Ashlesha Kumar, Vibhav Vineet +1
Self-supervised learning has emerged as a powerful paradigm for label-free model pretraining, particularly in the video domain, where manual annotation is costly and time-intensive…
Inference-Time Scaling for Complex Tasks: Where We Stand and What Lies Ahead
Vidhisha Balachandran, Jingya Chen, Lingjiao Chen +8
Inference-time scaling can enhance the reasoning capabilities of large language models (LLMs) on complex problems that benefit from step-by-step problem solving. Although lengtheni…
MM-GEN: Enhancing Task Performance Through Targeted Multimodal Data Curation
Siddharth Joshi, Besmira Nushi, Vidhisha Balachandran +4
Vision-language models (VLMs) are highly effective but often underperform on specialized tasks; for example, Llava-1.5 struggles with chart and diagram understanding due to scarce…
RiTTA: Modeling Event Relations in Text-to-Audio Generation
Yuhang He, Yash Jain, Xubo Liu +2
Despite significant advancements in Text-to-Audio (TTA) generation models achieving high-fidelity audio with fine-grained context understanding, they struggle to model the relation…
Unearthing Skill-Level Insights for Understanding Trade-Offs of Foundation Models
Mazda Moayeri, Vidhisha Balachandran, Varun Chandrasekaran +6
With models getting stronger, evaluations have grown more complex, testing multiple skills in one benchmark and even in the same instance at once. However, skill-wise performance i…
On Occlusions in Video Action Detection: Benchmark Datasets And Training Recipes
Rajat Modi, Vibhav Vineet, Yogesh Singh Rawat
This paper explores the impact of occlusions in video action detection. We facilitate this study by introducing five new benchmark datasets namely O-UCF and O-JHMDB consisting of s…