60 citations · 103 across the 10 of their papers we have counts for
11 papers
Kimi-Audio Technical Report
KimiTeam, Ding Ding, Zeqian Ju +37
We present Kimi-Audio, an open-source audio foundation model that excels in audio understanding, generation, and conversation. We detail the practices in building Kimi-Audio, inclu…
Kimi-VL Technical Report
Kimi Team, Angang Du, Bohong Yin +92
We present Kimi-VL, an efficient open-source Mixture-of-Experts (MoE) vision-language model (VLM) that offers advanced multimodal reasoning, long-context understanding, and strong…
Muon is Scalable for LLM Training
Jingyuan Liu, Jianlin Su, Xingcheng Yao +25
Recently, the Muon optimizer based on matrix orthogonalization has demonstrated strong results in training small-scale language models, but the scalability to larger models has not…
MoBA: Mixture of Block Attention for Long-Context LLMs
Enzhe Lu, Zhejun Jiang, Jingyuan Liu +22
Scaling the effective context length is essential for advancing large language models (LLMs) toward artificial general intelligence (AGI). However, the quadratic increase in comput…
Kimi k1.5: Scaling Reinforcement Learning with LLMs
Kimi Team, Angang Du, Bofei Gao +93
Language model pretraining with next token prediction has proved effective for scaling compute but is limited to the amount of available training data. Scaling reinforcement learni…
STORYWARS: A Dataset and Instruction Tuning Baselines for Collaborative Story Understanding and Generation
Yulun Du, Lydia Chilton
Collaborative stories, which are texts created through the collaborative efforts of multiple authors with different writing styles and intentions, pose unique challenges for NLP mo…
GPS: Genetic Prompt Search for Efficient Few-shot Learning
Hanwei Xu, Yujun Chen, Yulun Du +4
Prompt-based techniques have demostrated great potential for improving the few-shot generalization of pretrained language models. However, their performance heavily relies on the m…
ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization
Hanwei Xu, Yujun Chen, Yulun Du +4
We propose a multitask pretraining approach ZeroPrompt for zero-shot generalization, focusing on task scaling and zero-shot prompting. While previous models are trained on only a f…
Distribution Matching for Rationalization
Yongfeng Huang, Yujun Chen, Yulun Du +1
The task of rationalization aims to extract pieces of input text as rationales to justify neural network predictions on text classification tasks. By definition, rationales represe…
Multimodal Polynomial Fusion for Detecting Driver Distraction
Yulun Du, Chirag Raman, Alan W Black +2
Distracted driving is deadly, claiming 3,477 lives in the U.S. in 2015 alone. Although there has been a considerable amount of research on modeling the distracted behavior of drive…
Controllable Invariance through Adversarial Feature Learning
Qizhe Xie, Zihang Dai, Yulun Du +2
Learning meaningful representations that maintain the content necessary for a particular task while filtering away detrimental variations is a problem of great interest in machine…