Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
One Model, Two Roles: Emergent Specialization in a Shared Recurrent Transformer
Jucheng Shen, Barbara Su, Anastasios Kyrillidis
Can a shared-weight recurrent Transformer develop distinct internal roles without being partitioned into separate modules? We study this in Asymmetric Input Recurrence (AIR), a min…
cs.LG2026
AdaPaD: Adaptive Parallel Deflation for PEFT with Self-Correcting Rank Discovery
Barbara Su, Fangshuo Liao, Anastasios Kyrillidis
Fine-tuning large language models with LoRA requires choosing a rank r before training starts. Existing approaches either extract rank-1 components sequentially, freezing each comp…
cs.LG2025
TwIST: Rigging the Lottery in Transformers with Independent Subnetwork Training
Michael Menezes, Barbara Su, Xinze Feng +3
We introduce TwIST, a distributed training framework for efficient large language model (LLM) sparsification. TwIST trains multiple subnetworks in parallel, periodically aggregates…