5 papers
Grammar Search for Multi-Agent Systems
Mayank Singh, Vikas Yadav, Shiva Krishna Reddy Malay +4
Automatic search for Multi-Agent Systems has recently emerged as a key focus in agentic AI research. Several prior approaches have relied on LLM-based free-form search over the cod…
Augmenting LLM Reasoning with Dynamic Notes Writing for Complex QA
Rishabh Maheshwary, Masoud Hashemi, Khyati Mahajan +5
Iterative RAG for multi-hop question answering faces challenges with lengthy contexts and the buildup of irrelevant information. This hinders a model's capacity to process and reas…
DNR Bench: Benchmarking Over-Reasoning in Reasoning LLMs
Masoud Hashemi, Oluwanifemi Bamgbose, Sathwik Tejaswi Madhusudhan +3
Test-time scaling has significantly improved large language model performance, enabling deeper reasoning to solve complex problems. However, this increased reasoning capability als…
Revitalizing Saturated Benchmarks: A Weighted Metric Approach for Differentiating Large Language Model Performance
Bryan Etzine, Masoud Hashemi, Nishanth Madhusudhan +4
Existing benchmarks are becoming saturated and struggle to separate model performances due to factors like data contamination and advancing LLM capabilities. This paper introduces…
Prompting with Phonemes: Enhancing LLMs' Multilinguality for Non-Latin Script Languages
Hoang H Nguyen, Khyati Mahajan, Vikas Yadav +4
Although multilingual LLMs have achieved remarkable performance across benchmarks, we find they continue to underperform on non-Latin script languages across contemporary LLM famil…