14 citations · 70 across the 73 of their papers we have counts for
8 papers · 1 filter
On Evaluating the Integration of Reasoning and Action in LLM Agents with Database Question Answering
Linyong Nan, Ellen Zhang, Weijin Zou +3
This study introduces a new long-form database question answering dataset designed to evaluate how Large Language Models (LLMs) interact with a SQL interpreter. The task necessitat…
MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning
Xiangru Tang, Anni Zou, Zhuosheng Zhang +5
Large language models (LLMs), despite their remarkable progress across various general domains, encounter significant barriers in medicine and healthcare. This field faces unique c…
ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code
Xiangru Tang, Yuliang Liu, Zefan Cai +21
Despite Large Language Models (LLMs) like GPT-4 achieving impressive results in function-level code generation, they struggle with repository-scale code understanding (e.g., coming…
DocMath-Eval: Evaluating Math Reasoning Capabilities of LLMs in Understanding Long and Specialized Documents
Yilun Zhao, Yitao Long, Hongjun Liu +7
Recent LLMs have demonstrated remarkable performance in solving exam-like math word problems. However, the degree to which these numerical reasoning skills are effective in real-wo…
FinanceMath: Knowledge-Intensive Math Reasoning in Finance Domains
Yilun Zhao, Hongjun Liu, Yitao Long +3
We introduce FinanceMath, a novel benchmark designed to evaluate LLMs' capabilities in solving knowledge-intensive math reasoning problems. Compared to prior works, this study feat…
Investigating Data Contamination in Modern Benchmarks for Large Language Models
Chunyuan Deng, Yilun Zhao, Xiangru Tang +2
Recent observations have underscored a disparity between the inflated benchmark scores and the actual performance of LLMs, raising concerns about potential contamination of evaluat…
Struc-Bench: Are Large Language Models Really Good at Generating Complex Structured Data?
Xiangru Tang, Yiming Zong, Jason Phang +4
Despite the remarkable capabilities of Large Language Models (LLMs) like GPT-4, producing complex, structured tabular data remains challenging. Our study assesses LLMs' proficiency…
QTSumm: Query-Focused Summarization over Tabular Data
Yilun Zhao, Zhenting Qi, Linyong Nan +9
People primarily consult tables to conduct data analysis or answer specific questions. Text generation systems that can provide accurate table summaries tailored to users' informat…