activity
20172026
most citedThe GEM Benchmark: Natural Language Generation, its Evaluation and Metrics

52 citations · 62 across the 22 of their papers we have counts for

collaborators

28 papers

cs.AI2026

TeleTune: Evolving Agent Skills From Offline Telemetry

Justin Chih-Yao Chen, Elias Stengel-Eskin, Yan Chen +6

Computer-use agents need to capture procedural knowledge of how people use software. User telemetry offers a scalable source of this knowledge. However, learning reusable skills fr…

cs.LG2025

Sample-Efficient Online Learning in LM Agents via Hindsight Trajectory Rewriting

Michael Y. Hu, Benjamin Van Durme, Jacob Andreas +1

Language model (LM) agents deployed in novel environments often exhibit poor sample efficiency when learning from sequential interactions. This significantly hinders the usefulness…

cs.CL2025

LM Agents for Coordinating Multi-User Information Gathering

Harsh Jhamtani, Jacob Andreas, Benjamin Van Durme

This paper introduces PeopleJoin, a benchmark for evaluating LM-mediated collaborative problem solving. Given a user request, PeopleJoin agents must identify teammates who might be…

cs.CL2024

Towards Robust Evaluation of Unlearning in LLMs via Data Transformations

Abhinav Joshi, Shaswati Saha, Divyaksh Shukla +4

Large Language Models (LLMs) have shown to be a great success in a wide range of applications ranging from regular NLP-based use cases to AI agents. LLMs have been trained on a vas…

cs.CL2024

Steering Large Language Models between Code Execution and Textual Reasoning

Yongchao Chen, Harsh Jhamtani, Srinagesh Sharma +2

While a lot of recent research focuses on enhancing the textual reasoning capabilities of Large Language Models (LLMs) by optimizing the multi-agent framework or reasoning chains,…

cs.CL2024

Learning to Retrieve Iteratively for In-Context Learning

Yunmo Chen, Tongfei Chen, Harsh Jhamtani +4

We introduce iterative retrieval, a novel framework that empowers retrievers to make iterative decisions through policy optimization. Finding an optimal portfolio of retrieved item…