6 papers
Kimi K3: Open Frontier Intelligence
Kimi Team, Tongtong Bai, Yifan Bai +398
We introduce Kimi K3, a 2.8T parameter Mixture-of-Experts model with 104 billion activated parameters, native vision capabilities, and a 1-million-token context window. Kimi K3 is…
Provable Knowledge Acquisition and Extraction in One-Layer Transformers
Ruichen Xu, Kexin Chen
Large language models may encounter factual knowledge during pre-training yet fail to reliably use that knowledge after fine-tuning. Despite growing empirical evidence that MLP lay…
Generative AI as Digital Representatives in Collective Decision-Making: A Game-Theoretical Approach
Kexin Chen, Jianwei Huang, Yuan Luo
Generative Artificial Intelligence (GenAI) enables digital representatives to make decisions on behalf of team members in collaborative tasks, but faces challenges in accurately re…
Rethinking Benign Overfitting in Two-Layer Neural Networks
Ruichen Xu, Kexin Chen
Recent theoretical studies (Kou et al., 2023; Cao et al., 2022) have revealed a sharp phase transition from benign to harmful overfitting when the noise-to-feature ratio exceeds a…
General Scales Unlock AI Evaluation with Explanatory and Predictive Power
Lexin Zhou, Lorenzo Pacchiardi, Fernando MartÃnez-Plumed +23
Ensuring safe and effective use of AI requires understanding and anticipating its performance on novel tasks, from advanced scientific challenges to transformed workplace activitie…
Exploring Complex Mental Health Symptoms via Classifying Social Media Data with Explainable LLMs
Kexin Chen, Noelle Lim, Claire Lee +1
We propose a pipeline for gaining insights into complex diseases by training LLMs on challenging social media text data classification tasks, obtaining explanations for the classif…