8 papers
CUA-Skill: Develop Skills for Computer Using Agent
Tianyi Chen, Yinheng Li, Michael Solodko +12
Computer-Using Agents (CUAs) aim to autonomously operate computer systems to complete real-world tasks. However, existing agentic systems remain difficult to scale and lag behind h…
AppSelectBench: Application-Level Tool Selection Benchmark
Tianyi Chen, Michael Solodko, Sen Wang +14
Computer Using Agents (CUAs) are increasingly equipped with external tools, enabling them to perform complex and realistic tasks. For CUAs to operate effectively, application selec…
Hierarchical Self-Attention: Generalizing Neural Attention Mechanics to Multi-Scale Problems
Saeed Amizadeh, Sara Abdali, Yinheng Li +1
Transformers and their attention mechanism have been revolutionary in the field of Machine Learning. While originally proposed for the language data, they quickly found their way t…
Instruction Agent: Enhancing Agent with Expert Demonstration
Yinheng Li, Hailey Hultquist, Justin Wagle +1
Graphical user interface (GUI) agents have advanced rapidly but still struggle with complex tasks involving novel UI elements, long-horizon actions, and personalized trajectories.…
VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Tasks
Lawrence Jang, Yinheng Li, Dan Zhao +5
Videos are often used to learn or extract the necessary information to complete tasks in ways different than what text and static imagery alone can provide. However, many existing…
WinClick: GUI Grounding with Multimodal Large Language Models
Zheng Hui, Yinheng Li, Dan zhao +3
Graphical User Interface (GUI) tasks are vital for automating workflows such as software testing, user interface navigation. For users, the GUI is the most intuitive platform for i…