8 papers
Self-Guided Test-Time Training for Long-Context LLMs
Xinyu Zhu, Zhe Xu, Xiaohan Wei +10
Long-context processing has become increasingly important for large language models (LLMs), but simply extending the context window does not guarantee effective utilization of long…
Do LLM Evaluators Prefer Themselves for a Reason?
Wei-Lin Chen, Zhepei Wei, Xinyu Zhu +2
Large language models (LLMs) are increasingly used as automatic evaluators in applications such as benchmarking, reward modeling, and self-refinement. Prior work highlights a poten…
The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning
Xinyu Zhu, Mengzhou Xia, Zhepei Wei +3
Reinforcement learning with verifiable rewards (RLVR) is a promising approach for training language models (LMs) on reasoning tasks that elicit emergent long chains of thought (CoT…
AdaDecode: Accelerating LLM Decoding with Adaptive Layer Parallelism
Zhepei Wei, Wei-Lin Chen, Xinyu Zhu +1
Large language models (LLMs) are increasingly used for long-content generation (e.g., long Chain-of-Thought reasoning) where decoding efficiency becomes a critical bottleneck: Auto…
InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationales
Zhepei Wei, Wei-Lin Chen, Yu Meng
Retrieval-augmented generation (RAG) has shown promising potential to enhance the accuracy and factuality of language models (LMs). However, imperfect retrievers or noisy corpora c…
Seed-Guided Topic Discovery with Out-of-Vocabulary Seeds
Yu Zhang, Yu Meng, Xuan Wang +2
Discovering latent topics from text corpora has been studied for decades. Many existing topic models adopt a fully unsupervised setting, and their discovered topics may not cater t…