40 citations · 62 across the 7 of their papers we have counts for
8 papers
Evaluating LLMs as Agentic Workflow Executors in Real-World Customer Support Skills
Md Tahmid Rahman Laskar, Gundeep Singh, Xue-Yong Fu +11
Enterprise customer-support agents increasingly rely on Large Language Models (LLMs) to execute bounded, tool-using workflows. We study this problem through the lens of Skill LLMs:…
When Better Turns Do Not Make Better Agents: Diagnosing the Gap Between Next-Turn Metrics and Workflow Success
Md Tahmid Rahman Laskar, Xue-Yong Fu, Gundeep Singh +11
Agent models are frequently evaluated one decision at a time, where the model predicts the next action based on the gold interaction history, which is scored against a reference. W…
From Text to Voice: A Reproducible and Verifiable Framework for Evaluating Tool Calling LLM Agents
Md Tahmid Rahman Laskar, Xue-Yong Fu, Seyyed Saeed Sarfjoo +3
Voice agents increasingly require reliable tool use from speech, whereas prominent tool-calling benchmarks remain text-based. We study whether verified text benchmarks can be conve…
Style-agnostic evaluation of ASR using multiple reference transcripts
Quinten McNamara, Miguel Ángel del Río Fernández, Nishchal Bhandari +4
Word error rate (WER) as a metric has a variety of limitations that have plagued the field of speech recognition. Evaluation datasets suffer from varying style, formality, and inhe…
Reverb: Open-Source ASR and Diarization from Rev
Nishchal Bhandari, Danny Chen, Miguel Ángel del Río Fernández +10
Today, we are open-sourcing our core speech recognition and diarization models for non-commercial use. We are releasing both a full production pipeline for developers as well as pa…
Earnings-22: A Practical Benchmark for Accents in the Wild
Miguel Del Rio, Peter Ha, Quinten McNamara +2
Modern automatic speech recognition (ASR) systems have achieved superhuman Word Error Rate (WER) on many common corpora despite lacking adequate performance on speech in the wild.…
Accented Speech Recognition: A Survey
Arthur Hinsvark, Natalie Delworth, Miguel Del Rio +9
Automatic Speech Recognition (ASR) systems generalize poorly on accented speech. The phonetic and linguistic variability of accents present hard challenges for ASR systems today in…
Earnings-21: A Practical Benchmark for ASR in the Wild
Miguel Del Rio, Natalie Delworth, Ryan Westerman +7
Commonly used speech corpora inadequately challenge academic and commercial ASR systems. In particular, speech corpora lack metadata needed for detailed analysis and WER measuremen…