22 citations · 100 across the 15 of their papers we have counts for
5 papers · 1 filter
Automated Few-shot Classification with Instruction-Finetuned Language Models
Rami Aly, Xingjian Shi, Kaixiang Lin +2
A particularly successful class of approaches for few-shot learning combines language models with prompts -- hand-crafted task descriptions that complement data samples. However, d…
Tailoring Instructions to Student's Learning Levels Boosts Knowledge Distillation
Yuxin Ren, Zihan Zhong, Xingjian Shi +3
It has been commonly observed that a teacher model with superior performance does not necessarily result in a stronger student, highlighting a discrepancy between current teacher t…
Parameter-Efficient Fine-Tuning Design Spaces
Jiaao Chen, Aston Zhang, Xingjian Shi +3
Parameter-efficient fine-tuning aims to achieve performance comparable to fine-tuning, using fewer trainable parameters. Several strategies (e.g., Adapters, prefix tuning, BitFit,…
SPT: Semi-Parametric Prompt Tuning for Multitask Prompted Learning
M Saiful Bari, Aston Zhang, Shuai Zheng +4
Pre-trained large language models can efficiently interpolate human-written prompts in a natural way. Multitask prompted learning can help generalization through a diverse set of t…
Distiller: A Systematic Study of Model Distillation Methods in Natural Language Processing
Haoyu He, Xingjian Shi, Jonas Mueller +3
We aim to identify how different components in the KD pipeline affect the resulting performance and how much the optimal KD pipeline varies across different datasets/tasks, such as…