53 citations · 98 across the 16 of their papers we have counts for
13 papers · 1 filter
Synthesize and Reward -- Reinforcement Learning for Multi-Step Tool Use in Live Environments
Ibrahim Abdelaziz, Asim Munawar, Kinjal Basu +4
Training LLMs to orchestrate multi-step tool calls is held back by three coupled obstacles: realistic stateful execution environments are costly to build, synthetic training querie…
Simulating Complex Multi-Turn Tool Calling Interactions in Stateless Execution Environments
Maxwell Crouse, Ibrahim Abdelaziz, Kshitij Fadnis +6
Synthetic data has proven itself to be a valuable resource for tuning smaller, cost-effective language models to handle the complexities of multi-turn tool calling conversations. W…
MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systems
Yannis Katsis, Sara Rosenthal, Kshitij Fadnis +7
Retrieval-augmented generation (RAG) has recently become a very popular task for Large Language Models (LLMs). Evaluating them on multi-turn RAG conversations, where the system is…
Reducing the Scope of Language Models
David Yunis, Siyu Huo, Chulaka Gunasekara +1
Large language models (LLMs) are deployed in a wide variety of user-facing applications. Typically, these deployments have some specific purpose, like answering questions grounded…
Multi-Document Grounded Multi-Turn Synthetic Dialog Generation
Young-Suk Lee, Chulaka Gunasekara, Danish Contractor +2
We introduce a technique for multi-document grounded multi-turn synthetic dialog generation that incorporates three main ideas. First, we control the overall dialog flow using taxo…
Evaluating Robustness of Dialogue Summarization Models in the Presence of Naturally Occurring Variations
Ankita Gupta, Chulaka Gunasekara, Hui Wan +3
Dialogue summarization task involves summarizing long conversations while preserving the most salient information. Real-life dialogues often involve naturally occurring variations…
MISMATCH: Fine-grained Evaluation of Machine-generated Text with Mismatch Error Types
Keerthiram Murugesan, Sarathkrishna Swaminathan, Soham Dan +9
With the growing interest in large language models, the need for evaluating the quality of machine text compared to reference (typically human-generated) text has become focal atte…
The Benefits of Bad Advice: Autocontrastive Decoding across Model Layers
Ariel Gera, Roni Friedman, Ofir Arviv +4
Applying language models to natural language processing tasks typically relies on the representations in the final model layer, as intermediate hidden layer representations are pre…
Heuristic-based Inter-training to Improve Few-shot Multi-perspective Dialog Summarization
Benjamin Sznajder, Chulaka Gunasekara, Guy Lev +3
Many organizations require their customer-care agents to manually summarize their conversations with customers. These summaries are vital for decision making purposes of the organi…
Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks
Hanjie Chen, Song Feng, Jatin Ganhotra +4
Explaining neural network models is important for increasing their trustworthiness in real-world applications. Most existing methods generate post-hoc explanations for neural netwo…
Overview of the Ninth Dialog System Technology Challenge: DSTC9
Chulaka Gunasekara, Seokhwan Kim, Luis Fernando D'Haro +36
This paper introduces the Ninth Dialog System Technology Challenge (DSTC-9). This edition of the DSTC focuses on applying end-to-end dialog technologies for four distinct tasks in…
Conversational Document Prediction to Assist Customer Care Agents
Jatin Ganhotra, Haggai Roitman, Doron Cohen +6
A frequent pattern in customer care conversations is the agents responding with appropriate webpage URLs that address users' needs. We study the task of predicting the documents th…
The Eighth Dialog System Technology Challenge
Seokhwan Kim, Michel Galley, Chulaka Gunasekara +18
This paper introduces the Eighth Dialog System Technology Challenge. In line with recent challenges, the eighth edition focuses on applying end-to-end dialog technologies in a prag…