1 citations · 1 across the 9 of their papers we have counts for
5 papers · 1 filter
Through the Valley: Path to Effective Long CoT Training for Small Language Models
Renjie Luo, Jiaxi Li, Chen Huang +1
Long chain-of-thought (CoT) supervision has become a common strategy to enhance reasoning in language models. While effective for large models, we identify a phenomenon we call Lon…
WildLong: Synthesizing Realistic Long-Context Instruction Data at Scale
Jiaxi Li, Xingxing Zhang, Xun Wang +6
Large language models (LLMs) with extended context windows enable tasks requiring extensive information integration but are limited by the scarcity of high-quality, diverse dataset…
Unraveling Feature Extraction Mechanisms in Neural Networks
Xiaobing Sun, Jiaxi Li, Wei Lu
The underlying mechanism of neural networks in capturing precise knowledge has been the subject of consistent research efforts. In this work, we propose a theoretical approach base…
Decomposed Prompt Tuning via Low-Rank Reparameterization
Yao Xiao, Lu Xu, Jiaxi Li +2
While prompt tuning approaches have achieved competitive performance with high efficiency, we observe that they invariably employ the same initialization process, wherein the soft…
Contextual Distortion Reveals Constituency: Masked Language Models are Implicit Parsers
Jiaxi Li, Wei Lu
Recent advancements in pre-trained language models (PLMs) have demonstrated that these models possess some degree of syntactic awareness. To leverage this knowledge, we propose a n…