35 citations · 102 across the 10 of their papers we have counts for
9 papers · 1 filter
AsyncOPD: How Stale Can On-Policy Distillation Be?
Wonjun Kang, Kevin Galim, Seunghyuk Oh +9
On-policy distillation (OPD) trains a student on its own rollouts guided by teacher feedback and is becoming increasingly important for large language model (LLM) post-training. Li…
You Don't Need to Run Every Eval
Yuchen Zeng, Dimitris Papailiopoulos
A modern model release reports scores on 40+ benchmarks and the same evaluations were run many more times before it: to track training progress, compare design choices, and select…
Parameter-Efficient Fine-Tuning of State Space Models
Kevin Galim, Wonjun Kang, Yuchen Zeng +2
Deep State Space Models (SSMs), such as Mamba (Gu & Dao, 2024), have become powerful tools for language modeling, offering high performance and linear scalability with sequence len…
Can MLLMs Perform Text-to-Image In-Context Learning?
Yuchen Zeng, Wonjun Kang, Yicong Chen +2
The evolution from Large Language Models (LLMs) to Multimodal Large Language Models (MLLMs) has spurred research into extending In-Context Learning (ICL) to its multimodal counterp…
The Expressive Power of Low-Rank Adaptation
Yuchen Zeng, Kangwook Lee
Low-Rank Adaptation (LoRA), a parameter-efficient fine-tuning method that leverages low-rank adaptation of weight matrices, has emerged as a prevalent technique for fine-tuning pre…
Outlier-Robust Group Inference via Gradient Space Clustering
Yuchen Zeng, Kristjan Greenewald, Kangwook Lee +2
Traditional machine learning models focus on achieving good performance on the overall training distribution, but they often underperform on minority groups. Existing methods can i…
Equal Improvability: A New Fairness Notion Considering the Long-term Impact
Ozgur Guldogan, Yuchen Zeng, Jy-yong Sohn +2
Devising a fair classifier that does not discriminate against different groups is an important problem in machine learning. Although researchers have proposed various ways of defin…
LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Tasks
Tuan Dinh, Yuchen Zeng, Ruisu Zhang +6
Fine-tuning pretrained language models (LMs) without making any architectural changes has become a norm for learning various language downstream tasks. However, for non-language do…
Improving Fairness via Federated Learning
Yuchen Zeng, Hongxu Chen, Kangwook Lee
Recently, lots of algorithms have been proposed for learning a fair classifier from decentralized data. However, many theoretical and algorithmic questions remain open. First, is f…