563 citations · 655 across the 15 of their papers we have counts for
11 papers · 1 filter
Thinking at the Right Size: Amortized Distillation Across Post-Trained LLMs
Yan Zhou, Sara Kangaslahti, Jonathan Geuter +4
Practical deployment of large language models (LLMs) requires families of post-trained variants---instruction-tuned, reasoning-tuned, and chat-style models---each at multiple sizes…
Understanding Layer Patching in Model Size Interpolation
Sara Kangaslahti, Jonathan Geuter, Nihal V. Nayak +3
Zero-shot model size interpolation aims to create new models of intermediate target sizes by combining existing models without additional training. Recent work on boomerang distill…
A Critical Look at Targeted Instruction Selection: Disentangling What Matters (and What Doesn't)
Nihal V. Nayak, Paula Rodriguez-Diaz, Neha Hulkund +2
Instruction fine-tuning of large language models (LLMs) often involves selecting a subset of instruction training data from a large candidate pool, using a small query set from the…
Boomerang Distillation Enables Zero-Shot Model Size Interpolation
Sara Kangaslahti, Nihal V. Nayak, Jonathan Geuter +3
Large language models (LLMs) are typically deployed under diverse memory and compute constraints. Existing approaches build model families by training each size independently, whic…
K-Paths: Reasoning over Graph Paths for Drug Repurposing and Drug Interaction Prediction
Tassallah Abdullahi, Ioanna Gemou, Nihal V. Nayak +4
Biomedical knowledge graphs (KGs) encode rich, structured information critical for drug discovery tasks, but extracting meaningful insights from large-scale KGs remains challenging…
CEREAL: Few-Sample Clustering Evaluation
Nihal V. Nayak, Ethan R. Elenberg, Clemens Rosenbaum
Evaluating clustering quality with reliable evaluation metrics like normalized mutual information (NMI) requires labeled data that can be expensive to annotate. We focus on the und…
Learning to Compose Soft Prompts for Compositional Zero-Shot Learning
Nihal V. Nayak, Peilin Yu, Stephen H. Bach
We introduce compositional soft prompting (CSP), a parameter-efficient learning technique to improve the zero-shot compositionality of large-scale pretrained vision-language models…
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts
Stephen H. Bach, Victor Sanh, Zheng-Xin Yong +24
PromptSource is a system for creating, sharing, and using natural language prompts. Prompts are functions that map an example from a dataset to a natural language input and target…
TAGLETS: A System for Automatic Semi-Supervised Learning with Auxiliary Data
Wasu Piriyakulkij, Cristina Menghini, Ross Briden +4
Machine learning practitioners often have access to a spectrum of data: labeled data for the target task (which is often limited), unlabeled data, and auxiliary data, the many avai…
Multitask Prompted Training Enables Zero-Shot Task Generalization
Victor Sanh, Albert Webson, Colin Raffel +38
Large language models have recently been shown to attain reasonable zero-shot generalization on a diverse set of tasks (Brown et al., 2020). It has been hypothesized that this is a…
Zero-Shot Learning with Common Sense Knowledge Graphs
Nihal V. Nayak, Stephen H. Bach
Zero-shot learning relies on semantic class representations such as hand-engineered attributes or learned embeddings to predict classes without any labeled examples. We propose to…