35 citations · 102 across the 10 of their papers we have counts for
10 papers
AsyncOPD: How Stale Can On-Policy Distillation Be?
Wonjun Kang, Kevin Galim, Seunghyuk Oh +9
On-policy distillation (OPD) trains a student on its own rollouts guided by teacher feedback and is becoming increasingly important for large language model (LLM) post-training. Li…
You Don't Need to Run Every Eval
Yuchen Zeng, Dimitris Papailiopoulos
A modern model release reports scores on 40+ benchmarks and the same evaluations were run many more times before it: to track training progress, compare design choices, and select…
Parameter-Efficient Fine-Tuning of State Space Models
Kevin Galim, Wonjun Kang, Yuchen Zeng +2
Deep State Space Models (SSMs), such as Mamba (Gu & Dao, 2024), have become powerful tools for language modeling, offering high performance and linear scalability with sequence len…
Can MLLMs Perform Text-to-Image In-Context Learning?
Yuchen Zeng, Wonjun Kang, Yicong Chen +2
The evolution from Large Language Models (LLMs) to Multimodal Large Language Models (MLLMs) has spurred research into extending In-Context Learning (ICL) to its multimodal counterp…
The Expressive Power of Low-Rank Adaptation
Yuchen Zeng, Kangwook Lee
Low-Rank Adaptation (LoRA), a parameter-efficient fine-tuning method that leverages low-rank adaptation of weight matrices, has emerged as a prevalent technique for fine-tuning pre…
Outlier-Robust Group Inference via Gradient Space Clustering
Yuchen Zeng, Kristjan Greenewald, Kangwook Lee +2
Traditional machine learning models focus on achieving good performance on the overall training distribution, but they often underperform on minority groups. Existing methods can i…
Equal Improvability: A New Fairness Notion Considering the Long-term Impact
Ozgur Guldogan, Yuchen Zeng, Jy-yong Sohn +2
Devising a fair classifier that does not discriminate against different groups is an important problem in machine learning. Although researchers have proposed various ways of defin…
LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Tasks
Tuan Dinh, Yuchen Zeng, Ruisu Zhang +6
Fine-tuning pretrained language models (LMs) without making any architectural changes has become a norm for learning various language downstream tasks. However, for non-language do…
Improving Fairness via Federated Learning
Yuchen Zeng, Hongxu Chen, Kangwook Lee
Recently, lots of algorithms have been proposed for learning a fair classifier from decentralized data. However, many theoretical and algorithmic questions remain open. First, is f…
Multiway clustering via tensor block models
Miaoyan Wang, Yuchen Zeng
We consider the problem of identifying multiway block structure from a large noisy tensor. Such problems arise frequently in applications such as genomics, recommendation system, t…