2 papers
cs.LG2025
Llamba: Scaling Distilled Recurrent Models for Efficient Language Processing
Aviv Bick, Tobias Katsch, Nimit Sohoni +2
We introduce Llamba, a family of efficient recurrent language models distilled from Llama-3.x into the Mamba architecture. The series includes Llamba-1B, Llamba-3B, and Llamba-8B,…
cs.LG2024
Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations
Michael Zhang, Nimit S. Sohoni, Hongyang R. Zhang +2
Spurious correlations pose a major challenge for robust machine learning. Models trained with empirical risk minimization (ERM) may learn to rely on correlations between class labe…