3 citations · 6 across the 9 of their papers we have counts for
9 papers
Do Transformers Have the Ability for Periodicity Generalization?
Huanyu Liu, Ge Li, Yihong Dong +7
Large language models (LLMs) based on the Transformer have demonstrated strong performance across diverse tasks. However, current models still exhibit substantial limitations in ou…
SMEC: Rethinking Matryoshka Representation Learning for Retrieval Embedding Compression
Biao Zhang, Lixin Chen, Tong Liu +1
Large language models (LLMs) generate high-dimensional embeddings that capture rich semantic and syntactic information. However, high-dimensional embeddings exacerbate computationa…
FrontendBench: A Benchmark for Evaluating LLMs on Front-End Development via Automatic Evaluation
Hongda Zhu, Yiwen Zhang, Bing Zhao +6
Large Language Models (LLMs) have made significant strides in front-end code generation. However, existing benchmarks exhibit several critical limitations: many tasks are overly si…
Taming the Titans: A Survey of Efficient LLM Inference Serving
Ranran Zhen, Juntao Li, Yixin Ji +7
Large Language Models (LLMs) for Generative AI have achieved remarkable progress, evolving into sophisticated and versatile tools widely adopted across various domains and applicat…
Kimi-Audio Technical Report
KimiTeam, Ding Ding, Zeqian Ju +37
We present Kimi-Audio, an open-source audio foundation model that excels in audio understanding, generation, and conversation. We detail the practices in building Kimi-Audio, inclu…
MoE Parallel Folding: Heterogeneous Parallelism Mappings for Efficient Large-Scale MoE Model Training with Megatron Core
Dennis Liu, Zijie Yan, Xin Yao +15
Mixture of Experts (MoE) models enhance neural network scalability by dynamically selecting relevant experts per input token, enabling larger model sizes while maintaining manageab…
Large Language Models for Traffic and Transportation Research: Methodologies, State of the Art, and Future Opportunities
Yimo Yan, Yejia Liao, Guanhao Xu +13
The rapid rise of Large Language Models (LLMs) is transforming traffic and transportation research, with significant advancements emerging between the years 2023 and 2025 -- a peri…
PerfSeer: An Efficient and Accurate Deep Learning Models Performance Predictor
Xinlong Zhao, Jiande Sun, Jia Zhang +4
Predicting the performance of deep learning (DL) models, such as execution time and resource utilization, is crucial for Neural Architecture Search (NAS), DL cluster schedulers, an…
Upcycling Large Language Models into Mixture of Experts
Ethan He, Abhinav Khattar, Ryan Prenger +7
Upcycling pre-trained dense language models into sparse mixture-of-experts (MoE) models is an efficient approach to increase the model capacity of already trained models. However,…