21 papers
Identity-Paired Progressive Depth Training: When Trainability Persists Beyond Expressibility
Athanasios Hadjidimoulas, Tirthak Patel, Anastasios Kyrillidis
Variational Quantum Algorithms (VQAs) are a leading paradigm for near-term quantum computing, yet their training suffers from sensitivity to circuit depth, initialization, and land…
SuperThoughts: Reasoning Tokens in Superposition
Zheyang Xiong, Shivam Garg, Max Yu +4
Long Chain-of-Thought (CoT) reasoning improves LLM problem-solving but is computationally expensive due to sequential token generation. While recent works explore reasoning in cont…
One Model, Two Roles: Emergent Specialization in a Shared Recurrent Transformer
Jucheng Shen, Barbara Su, Anastasios Kyrillidis
Can a shared-weight recurrent Transformer develop distinct internal roles without being partitioned into separate modules? We study this in Asymmetric Input Recurrence (AIR), a min…
AdaPaD: Adaptive Parallel Deflation for PEFT with Self-Correcting Rank Discovery
Barbara Su, Fangshuo Liao, Anastasios Kyrillidis
Fine-tuning large language models with LoRA requires choosing a rank r before training starts. Existing approaches either extract rank-1 components sequentially, freezing each comp…
SGD at the Edge of Stability: The Stochastic Sharpness Gap
Fangshuo Liao, Afroditi Kolomvaki, Anastasios Kyrillidis
When training neural networks with full-batch gradient descent (GD) and step size , the largest eigenvalue of the Hessian -- the sharpness -- rises to …
Convergence Analysis of Two-Layer Neural Networks under Gaussian Input Masking
Afroditi Kolomvaki, Fangshuo Liao, Evan Dramko +2
We investigate the convergence guarantee of two-layer neural network training with Gaussian randomly masked inputs. This scenario corresponds to Gaussian dropout at the input level…