6 papers
I-MCTS: Enhancing Agentic AutoML via Introspective Monte Carlo Tree Search
Zujie Liang, Feng Wei, Wujiang Xu +3
Recent advancements in large language models (LLMs) have shown remarkable potential in automating machine learning tasks. However, existing LLM-based agents often struggle with low…
Past Meets Present: Creating Historical Analogy with Large Language Models
Nianqi Li, Siyu Yuan, Jiangjie Chen +5
Historical analogies, which compare known past events with contemporary but unfamiliar events, are important abilities that help people make decisions and understand the world. How…
PowerAttention: Exponentially Scaling of Receptive Fields for Effective Sparse Attention
Lida Chen, Dong Xu, Chenxin An +8
Large Language Models (LLMs) face efficiency bottlenecks due to the quadratic complexity of the attention mechanism when processing long contexts. Sparse attention methods offer a…
QUILL: Quotation Generation Enhancement of Large Language Models
Jin Xiao, Bowei Zhang, Qianyu He +6
While Large language models (LLMs) have become excellent writing assistants, they still struggle with quotation generation. This is because they either hallucinate when providing f…
MultiLingPoT: Enhancing Mathematical Reasoning with Multilingual Program Fine-tuning
Nianqi Li, Zujie Liang, Siyu Yuan +3
Program-of-Thought (PoT), which aims to use programming language instead of natural language as an intermediate step in reasoning, is an important way for LLMs to solve mathematica…
SEGMENT+: Long Text Processing with Short-Context Language Models
Wei Shi, Shuang Li, Kerun Yu +9
There is a growing interest in expanding the input capacity of language models (LMs) across various domains. However, simply increasing the context window does not guarantee robust…