2 papers
cs.LG2026
Width and Depth Limits Commute During Training in Linear Residual Networks
Ruijia Zhang, Soufiane Hayou
Recent work established that the infinite-width and infinite-depth limits of residual networks commute at initialization when residual branches are scaled by . In p…
cs.LG2026
Geometry-Preserving Orthonormal Initialization for Low-Rank Adaptation in RLVR
Ruijia Zhang, Jiacheng Zhu, Hanqing Zhu +1
Low-rank adaptation (LoRA) and its variants enable parameter-efficient fine-tuning of large language models under the supervised fine-tuning (SFT) paradigm. However, their efficacy…