36 citations · 67 across the 7 of their papers we have counts for
7 papers
Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models
Haoran Li, Qingxiu Dong, Zhengyang Tang +17
We introduce Generalized Instruction Tuning (called GLAN), a general and scalable method for instruction tuning of Large Language Models (LLMs). Unlike prior work that relies on se…
Unleashing the potential of GNNs via Bi-directional Knowledge Transfer
Shuai Zheng, Zhizhe Liu, Zhenfeng Zhu +3
Based on the message-passing paradigm, there has been an amount of research proposing diverse and impressive feature propagation mechanisms to improve the performance of GNNs. Howe…
Tuna: Instruction Tuning using Feedback from Large Language Models
Haoran Li, Yiran Liu, Xingxing Zhang +2
Instruction tuning of open-source large language models (LLMs) like LLaMA, using direct outputs from more powerful LLMs such as Instruct-GPT and GPT-4, has proven to be a cost-effe…
Overcoming Recency Bias of Normalization Statistics in Continual Learning: Balance and Adaptation
Yilin Lyu, Liyuan Wang, Xingxing Zhang +4
Continual learning entails learning a sequence of tasks and balancing their knowledge appropriately. With limited access to old training samples, much of the current work in deep n…
Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimality
Liyuan Wang, Jingyi Xie, Xingxing Zhang +3
Prompt-based continual learning is an emerging direction in leveraging pre-trained knowledge for downstream continual learning, and has almost reached the performance pinnacle unde…
LongNet: Scaling Transformers to 1,000,000,000 Tokens
Jiayu Ding, Shuming Ma, Li Dong +5
Scaling sequence length has become a critical demand in the era of large language models. However, existing methods struggle with either computational complexity or model expressiv…