2 citations · 5 across the 6 of their papers we have counts for
4 papers · 1 filter
A Mean Field Ansatz for Zero-Shot Weight Transfer
Xingyuan Chen, Wenwei Kuang, Lei Deng +3
The pre-training cost of large language models (LLMs) is prohibitive. One cutting-edge approach to reduce the cost is zero-shot weight transfer, also known as model growth for some…
Retrieval Meets Reasoning: Dynamic In-Context Editing for Long-Text Understanding
Weizhi Fei, Xueyan Niu, Guoqing Xie +4
Current Large Language Models (LLMs) face inherent limitations due to their pre-defined context lengths, which impede their capacity for multi-hop reasoning within extensive textua…
Beyond Scaling Laws: Understanding Transformer Performance with Associative Memory
Xueyan Niu, Bo Bai, Lei Deng +1
Increasing the size of a Transformer does not always lead to enhanced performance. This phenomenon cannot be explained by the empirical scaling laws. Furthermore, the model's enhan…
Extreme Video Compression with Pre-trained Diffusion Models
Bohan Li, Yiming Liu, Xueyan Niu +3
Diffusion models have achieved remarkable success in generating high quality image and video data. More recently, they have also been used for image compression with high perceptua…