4 papers · 1 filter
Decouple Searching from Training: Scaling Data Mixing via Model Merging for Large Language Model Pre-training
Shengrui Li, Fei Zhao, Kaiyan Zhao +6
Determining an effective data mixture is a key factor in Large Language Model (LLM) pre-training, where models must balance general competence with proficiency on hard tasks such a…
Derailer-Rerailer: Adaptive Verification for Efficient and Reliable Language Model Reasoning
Guangya Wan, Yuqi Wu, Hao Wang +3
Large Language Models (LLMs) have shown impressive reasoning capabilities, yet existing prompting methods face a critical trade-off: simple approaches often struggle with complex t…
Disparities in LLM Reasoning Accuracy and Explanations: A Case Study on African American English
Runtao Zhou, Guangya Wan, Saadia Gabriel +4
Large Language Models (LLMs) have demonstrated remarkable capabilities in reasoning tasks, leading to their widespread deployment. However, recent studies have highlighted concerni…
Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Sampling
Guangya Wan, Yuqi Wu, Jie Chen +1
Self-Consistency mitigates hallucinations in Large Language Models (LLMs) by sampling multiple reasoning paths,but it lacks a systematic approach to determine the optimal number of…