794 citations · 1.2k across the 30 of their papers we have counts for
5 papers · 1 filter
FLM-101B: An Open LLM and How to Train It with $100K Budget
Xiang Li, Yiqun Yao, Xin Jiang +10
Large language models (LLMs) are considered important approaches towards foundational machine intelligence, achieving remarkable success in Natural Language Processing and multimod…
Masked Structural Growth for 2x Faster Language Model Pre-training
Yiqun Yao, Zheng Zhang, Jing Li +1
Accelerating large language model pre-training is a critical issue in present research. In this paper, we focus on speeding up pre-training by progressively growing from a small Tr…
End-to-end contextual asr based on posterior distribution adaptation for hybrid ctc/attention system
Zhengyi Zhang, Pan Zhou
End-to-end (E2E) speech recognition architectures assemble all components of traditional speech recognition system into a single model. Although it simplifies ASR system, it introd…
A Dual-Channel Framework for Sarcasm Recognition by Detecting Sentiment Conflict
Yiyi Liu, Yequan Wang, Aixin Sun +3
Sarcasm employs ambivalence, where one says something positive but actually means negative, and vice versa. The essence of sarcasm, which is also a sufficient and necessary conditi…
Star-Transformer
Qipeng Guo, Xipeng Qiu, Pengfei Liu +3
Although Transformer has achieved great successes on many NLP tasks, its heavy structure with fully-connected attention connections leads to dependencies on large training data. In…