2 citations · 3 across the 11 of their papers we have counts for
5 papers · 1 filter
Through the Valley: Path to Effective Long CoT Training for Small Language Models
Renjie Luo, Jiaxi Li, Chen Huang +1
Long chain-of-thought (CoT) supervision has become a common strategy to enhance reasoning in language models. While effective for large models, we identify a phenomenon we call Lon…
From Tens of Hours to Tens of Thousands: Scaling Back-Translation for Speech Recognition
Tianduo Wang, Lu Xu, Wei Lu +1
Recent advances in Automatic Speech Recognition (ASR) have been largely fueled by massive speech corpora. However, extending coverage to diverse languages with limited resources re…
Self-Harmonized Chain of Thought
Ziqi Jin, Wei Lu
Chain-of-thought (CoT) prompting has demonstrated the capacity of large language models to perform complex reasoning through intermediate steps. While effective, current CoT method…
Unraveling Feature Extraction Mechanisms in Neural Networks
Xiaobing Sun, Jiaxi Li, Wei Lu
The underlying mechanism of neural networks in capturing precise knowledge has been the subject of consistent research efforts. In this work, we propose a theoretical approach base…
Decomposed Prompt Tuning via Low-Rank Reparameterization
Yao Xiao, Lu Xu, Jiaxi Li +2
While prompt tuning approaches have achieved competitive performance with high efficiency, we observe that they invariably employ the same initialization process, wherein the soft…