4 papers · 1 filter
VisualToolAgent (VisTA): A Reinforcement Learning Framework for Visual Tool Selection
Zeyi Huang, Yuyang Ji, Anirudh Sundara Rajan +5
We introduce VisTA, a new reinforcement learning framework that empowers visual agents to dynamically explore, select, and combine tools from a diverse library based on empirical p…
Do Vision Models Develop Human-Like Progressive Difficulty Understanding?
Zeyi Huang, Utkarsh Ojha, Yuyang Ji +2
When a human undertakes a test, their responses likely follow a pattern: if they answered an easy question incorrectly, they would likely answer a more difficult one…
IMPROVE: Iterative Model Pipeline Refinement and Optimization Leveraging LLM Experts
Eric Xue, Ke Chen, Zeyi Huang +2
Large language model (LLM) agents have emerged as a promising solution to automate the workflow of machine learning, but most existing methods share a common limitation: they attem…
Building a Mind Palace: Structuring Environment-Grounded Semantic Graphs for Effective Long Video Analysis with LLMs
Zeyi Huang, Yuyang Ji, Xiaofang Wang +11
Long-form video understanding with Large Vision Language Models is challenged by the need to analyze temporally dispersed yet spatially concentrated key moments within limited cont…