14 citations · 19 across the 9 of their papers we have counts for
5 papers · 1 filter
Bayesian Entropy-based Reordering for Calibrated Diffusion Language Models
Zhejun Jiang, Mijung Park
Masked Diffusion Language Models (MDLMs) generate sequences by iteratively replacing masked tokens with model predictions. At each denoising step, the decoder chooses which positio…
The Safety-Aware Denoiser for Text Diffusion Models
Amman Yusuf, Zhejun Jiang, Mijung Park
Recent work on text diffusion models offers a promising alternative to autoregressive generation, but controlling their safety remains underexplored. Existing safety approaches are…
Kimi K2: Open Agentic Intelligence
Kimi Team, Yifan Bai, Yiping Bao +195
We introduce Kimi K2, a Mixture-of-Experts (MoE) large language model with 32 billion activated parameters and 1 trillion total parameters. We propose the MuonClip optimizer, which…
Muon is Scalable for LLM Training
Jingyuan Liu, Jianlin Su, Xingcheng Yao +25
Recently, the Muon optimizer based on matrix orthogonalization has demonstrated strong results in training small-scale language models, but the scalability to larger models has not…
MoBA: Mixture of Block Attention for Long-Context LLMs
Enzhe Lu, Zhejun Jiang, Jingyuan Liu +22
Scaling the effective context length is essential for advancing large language models (LLMs) toward artificial general intelligence (AGI). However, the quadratic increase in comput…