9 citations · 9 across the 6 of their papers we have counts for
6 papers
LT2: Linear-Time Looped Transformers
Chunyuan Deng, Yizhe Zhang, Rui-Jie Zhu +4
Looped Transformers (LT) have emerged as a powerful architecture by iterating their layers multiple times before decoding the final token. However, pairing them with full attention…
Federation of Experts: Communication Efficient Distributed Inference for Large Language Models
Muhammad Shahir Abdurrahman, Chun Deng, Azalia Mirhoseini +1
Mixture of experts has emerged as the primary mechanism for making Large Language Models (LLMs) computationally efficient. However, in distributed settings, communicating token emb…
ByteFlow: Language Modeling through Adaptive Byte Compression without a Tokenizer
Chunyuan Deng, Sanket Lokegaonkar, Colin Lockard +3
Modern language models still rely on fixed, pre-defined subword tokenizations. Once a tokenizer is trained, the LM can only operate at this fixed level of granularity, which often…
Unveiling the Spectrum of Data Contamination in Language Models: A Survey from Detection to Remediation
Chunyuan Deng, Yilun Zhao, Yuzhao Heng +4
Data contamination has garnered increased attention in the era of large language models (LLMs) due to the reliance on extensive internet-derived training corpora. The issue of trai…
MIMIR: A Streamlined Platform for Personalized Agent Tuning in Domain Expertise
Chunyuan Deng, Xiangru Tang, Yilun Zhao +5
Recently, large language models (LLMs) have evolved into interactive agents, proficient in planning, tool use, and task execution across a wide variety of tasks. However, without s…
Investigating Data Contamination in Modern Benchmarks for Large Language Models
Chunyuan Deng, Yilun Zhao, Xiangru Tang +2
Recent observations have underscored a disparity between the inflated benchmark scores and the actual performance of LLMs, raising concerns about potential contamination of evaluat…