4 citations · 10 across the 26 of their papers we have counts for
8 papers · 1 filter
How Do Transformers Learn to Represent Symmetries?
Eduardo Santos-Escriche, Valerie Engelmayer, Ya-Wei Eileen Lin +1
Training Transformer-based architectures with finite data augmentation has become an increasingly popular approach in geometric machine learning. Despite its empirical success, the…
Training-Free Transformer Merging via Sequential Local Operator Alignment
Akansh Maurya, Ya-Wei Eileen Lin, Stefanie Jegelka +2
Training-free model merging aims to combine multiple fine-tuned models into a single model without further optimization on labeled data. Yet, in transformers, independently merging…
Efficient Learning and Symmetry Discovery under Exact Invariances
Ashkan Soleymani, Behrooz Tahmasebi, Patrick Jaillet +1
Learning with group invariances is central to many scientific and geometric learning problems, yet its computational foundations remain poorly understood. Even for classical superv…
Data Augmentation: A Fourier Analysis Perspective
Behrooz Tahmasebi, Melanie Weber, Stefanie Jegelka
Data augmentation is a simple and model-agnostic approach for exploiting known invariances in learning problems. Given a group acting on the input space, one augments the training…
No More K-means: Single-Stage Sparse Coding for Efficient Multi-Vector Retrieval
Lixuan Guo, Yifei Wang, Tiansheng Wen +3
Multi-vector retrieval (MVR) models, exemplified by ColBERT, have established new benchmarks in retrieval accuracy by preserving fine-grained token-level interactions. However, thi…
Reinforce Adjoint Matching: Scaling RL Post-Training of Diffusion and Flow-Matching Models
Andreas Bergmeister, Stefanie Jegelka, Nikolas Nüsken +2
Diffusion and flow-matching models scale because pretraining is supervised regression: a clean sample is noised analytically, and a model regresses against a closed-form target. RL…
Scaling Attention via Feature Sparsity
Yan Xie, Tiansheng Wen, Tangda Huang +4
Scaling Transformers to ultra-long contexts is bottlenecked by the cost of self-attention. Existing methods reduce this cost along the sequence axis through local window…
CSRv2: Unlocking Ultra-Sparse Embeddings
Lixuan Guo, Yifei Wang, Tiansheng Wen +5
In the era of large foundation models, the quality of embeddings has become a central determinant of downstream task performance and overall system capability. Yet widely used dens…