43 citations · 175 across the 12 of their papers we have counts for
12 papers
Llamba: Scaling Distilled Recurrent Models for Efficient Language Processing
Aviv Bick, Tobias Katsch, Nimit Sohoni +2
We introduce Llamba, a family of efficient recurrent language models distilled from Llama-3.x into the Mamba architecture. The series includes Llamba-1B, Llamba-3B, and Llamba-8B,…
Monarch: Expressive Structured Matrices for Efficient and Accurate Training
Tri Dao, Beidi Chen, Nimit Sohoni +7
Large neural networks excel in many domains, but they are expensive to train and fine-tune. A popular approach to reduce their compute or memory requirements is to replace dense we…
Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations
Michael Zhang, Nimit S. Sohoni, Hongyang R. Zhang +2
Spurious correlations pose a major challenge for robust machine learning. Models trained with empirical risk minimization (ERM) may learn to rely on correlations between class labe…
BARACK: Partially Supervised Group Robustness With Guarantees
Nimit S. Sohoni, Maziar Sanjabi, Nicolas Ballas +4
While neural networks have shown remarkable success on classification tasks in terms of average-case performance, they often fail to perform well on certain groups of the data. Suc…
Low-Shot Validation: Active Importance Sampling for Estimating Classifier Performance on Rare Categories
Fait Poms, Vishnu Sarukkai, Ravi Teja Mullapudi +4
For machine learning models trained with limited labeled training data, validation stands to become the main bottleneck to reducing overall annotation costs. We propose a statistic…
Mandoline: Model Evaluation under Distribution Shift
Mayee Chen, Karan Goel, Nimit S. Sohoni +3
Machine learning models are often deployed in different settings than they were trained and validated on, posing a challenge to practitioners who wish to predict how well the deplo…