43 citations · 60 across the 18 of their papers we have counts for
3 papers · 1 filter
Data-Efficient Adaptation of LLMs via Attention Head Reweighting
Tuomas Oikarinen, Zixiao Chen, Charlotte Siska +3
Learning effectively from limited data is critical in domains like security where labeled examples are scarce. Large language models (LLMs) have demonstrated some capabilities for…
Learning a Decision Tree Algorithm with Transformers
Yufan Zhuang, Liyuan Liu, Chandan Singh +2
Decision trees are renowned for their ability to achieve high predictive performance while remaining interpretable, especially on tabular data. Traditionally, they are constructed…
Sparse Backpropagation for MoE Training
Liyuan Liu, Jianfeng Gao, Weizhu Chen
One defining characteristic of Mixture-of-Expert (MoE) models is their capacity for conducting sparse computation via expert routing, leading to remarkable scalability. However, ba…