7 citations · 12 across the 15 of their papers we have counts for
8 papers · 1 filter
Explore Separately, Adopt Selectively: Execution-Guided Tool Use over Large Tool Libraries
Anna Mazhar, Seiji Maekawa, Nikita Bhutani
Large language model agents must discover relevant tools from large libraries, e.g., thousands of tools, as new needs arise during task execution. When several candidates could sat…
The Rarity Blind Spot: A Framework for Evaluating Statistical Reasoning in LLMs
Seiji Maekawa, Hayate Iso, Nikita Bhutani
Effective decision-making often relies on identifying what makes each candidate distinctive. While existing benchmarks for LLMs emphasize retrieving or summarizing information rele…
Towards Reliable Benchmarking: A Contamination Free, Controllable Evaluation Framework for Multi-step LLM Function Calling
Seiji Maekawa, Jackson Hassell, Pouya Pezeshkpour +2
Existing benchmarks for tool-augmented language models (TaLMs) lack fine-grained control over task difficulty and remain vulnerable to data contamination. We present FuncBenchGen,…
Same Content, Different Representations: A Controlled Study for Table QA
Yue Zhang, Seiji Maekawa, Nikita Bhutani
Table Question Answering (Table QA) in real-world settings must operate over both structured databases and semi-structured tables containing textual fields. However, existing bench…
Efficient Context Selection for Long-Context QA: No Tuning, No Iteration, Just Adaptive-
Chihiro Taguchi, Seiji Maekawa, Nikita Bhutani
Retrieval-augmented generation (RAG) and long-context language models (LCLMs) both address context limitations of LLMs in open-domain question answering (QA). However, optimal exte…
From Single to Multi: How LLMs Hallucinate in Multi-Document Summarization
Catarina G. Belem, Pouya Pezeshkpour, Hayate Iso +3
Although many studies have investigated and reduced hallucinations in large language models (LLMs) for single-document tasks, research on hallucination in multi-document summarizat…
Holistic Reasoning with Long-Context LMs: A Benchmark for Database Operations on Massive Textual Data
Seiji Maekawa, Hayate Iso, Nikita Bhutani
The rapid increase in textual information means we need more efficient methods to sift through, organize, and understand it all. While retrieval-augmented generation (RAG) models e…
Retrieval Helps or Hurts? A Deeper Dive into the Efficacy of Retrieval Augmentation to Language Models
Seiji Maekawa, Hayate Iso, Sairam Gurajada +1
While large language models (LMs) demonstrate remarkable performance, they encounter challenges in providing accurate responses when queried for information beyond their pre-traine…