6 citations · 18 across the 10 of their papers we have counts for
12 papers
AgentPerturb: Evaluating LLM Agent Robustness under Controlled Interaction Perturbations
Yuanjian Xu, Jinyi Han, Jianing Hao +5
Real-world tasks are often complex and noisy, posing substantial challenges for LLM agents. Yet existing evaluations lack a systematic framework for jointly characterizing potentia…
GELATO: Generative Entropy- and Lyapunov-based Adaptive Token Offloading for Device-Edge Speculative LLM Inference
Zengzipeng Tang, Yuxuan Sun, Wei Chen +2
The recent growth of on-device Large Language Model (LLM) inference has driven significant interest in device-edge collaborative LLM inference. As a promising architecture, Specula…
Computer Use at the Edge of the Statistical Precipice
Pierluca D'Oro, Sneha Silwal, William Wong +6
Evaluating Computer Use Agents (CUAs) on interactive environments is fraught with methodological pitfalls that the field has yet to systematically address. We show that a 1MB repla…
IDAT: A Multi-Modal Dataset and Toolkit for Building and Evaluating Interactive Task-Solving Agents
Shrestha Mohanty, Negar Arabzadeh, Andrea Tupini +5
Seamless interaction between AI agents and humans using natural language remains a key goal in AI research. This paper addresses the challenges of developing interactive agents cap…
A Data Source for Reasoning Embodied Agents
Jack Lanchantin, Sainbayar Sukhbaatar, Gabriel Synnaeve +3
Recent progress in using machine learning models for reasoning tasks has been driven by novel model architectures, large-scale pre-training protocols, and dedicated reasoning datas…
Transforming Human-Centered AI Collaboration: Redefining Embodied Agents Capabilities through Interactive Grounded Language Instructions
Shrestha Mohanty, Negar Arabzadeh, Julia Kiseleva +6
Human intelligence's adaptability is remarkable, allowing us to adjust to new tasks and multi-modal environments swiftly. This skill is evident from a young age as we acquire new a…
Collecting Interactive Multi-modal Datasets for Grounded Language Understanding
Shrestha Mohanty, Negar Arabzadeh, Milagro Teruel +9
Human intelligence can remarkably adapt quickly to new tasks and environments. Starting from a very young age, humans acquire new skills and learn how to solve new tasks either by…
IGLU 2022: Interactive Grounded Language Understanding in a Collaborative Environment at NeurIPS 2022
Julia Kiseleva, Alexey Skrynnik, Artem Zholus +14
Human intelligence has the remarkable ability to adapt to new tasks and environments quickly. Starting from a very young age, humans acquire new skills and learn how to solve new t…
Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021
Julia Kiseleva, Ziming Li, Mohammad Aliannejadi +18
Human intelligence has the remarkable ability to quickly adapt to new tasks and environments. Starting from a very young age, humans acquire new skills and learn how to solve new t…
Many Episode Learning in a Modular Embodied Agent via End-to-End Interaction
Yuxuan Sun, Ethan Carlson, Rebecca Qian +2
In this work we give a case study of an embodied machine-learning (ML) powered agent that improves itself via interactions with crowd-workers. The agent consists of a set of module…
NeurIPS 2021 Competition IGLU: Interactive Grounded Language Understanding in a Collaborative Environment
Julia Kiseleva, Ziming Li, Mohammad Aliannejadi +12
Human intelligence has the remarkable ability to adapt to new tasks and environments quickly. Starting from a very young age, humans acquire new skills and learn how to solve new t…
droidlet: modular, heterogenous, multi-modal agents
Anurag Pratik, Soumith Chintala, Kavya Srinet +10
In recent years, there have been significant advances in building end-to-end Machine Learning (ML) systems that learn at scale. But most of these systems are: (a) isolated (percept…