Showing 2024Show all
3 papers · 1 filter
cs.CL2024
AntLM: Bridging Causal and Masked Language Models
Xinru Yu, Bin Guo, Shiwei Luo +3
Causal Language Modeling (CLM) and Masked Language Modeling (MLM) are two mainstream learning paradigms based on Transformer networks, specifically the Decoder-only and Encoder-onl…
cs.LG2024
H-FedSN: Personalized Sparse Networks for Efficient and Accurate Hierarchical Federated Learning for IoT Applications
Jiechao Gao, Yuangang Li, Jie Wang +3
With the rapid development of the Internet of Things (IoT), federated learning (FL) has gained increasing attention for its privacy-preserving use of distributed data. However, con…
cs.CL2024
Length Generalization of Causal Transformers without Position Encoding
Jie Wang, Tao Ji, Yuanbin Wu +5
Generalizing to longer sentences is important for recent Transformer-based language models. Besides algorithms manipulating explicit position features, the success of Transformers…