4 papers
PowerAttention: Exponentially Scaling of Receptive Fields for Effective Sparse Attention
Lida Chen, Dong Xu, Chenxin An +8
Large Language Models (LLMs) face efficiency bottlenecks due to the quadratic complexity of the attention mechanism when processing long contexts. Sparse attention methods offer a…
I-MCTS: Enhancing Agentic AutoML via Introspective Monte Carlo Tree Search
Zujie Liang, Feng Wei, Wujiang Xu +3
Recent advancements in large language models (LLMs) have shown remarkable potential in automating machine learning tasks. However, existing LLM-based agents often struggle with low…
MultiLingPoT: Enhancing Mathematical Reasoning with Multilingual Program Fine-tuning
Nianqi Li, Zujie Liang, Siyu Yuan +3
Program-of-Thought (PoT), which aims to use programming language instead of natural language as an intermediate step in reasoning, is an important way for LLMs to solve mathematica…
QUILL: Quotation Generation Enhancement of Large Language Models
Jin Xiao, Bowei Zhang, Qianyu He +6
While Large language models (LLMs) have become excellent writing assistants, they still struggle with quotation generation. This is because they either hallucinate when providing f…