From the 1 of 13 linked papers with an AI index.
4 papers · 1 filter
OmniDelta: Skill-Driven Budget Allocation for Token Compression in OmniLLMs
Haoyang Huang, Wenjie Huang, Tianqi Xu +14
OmniDelta is a training-free framework that dynamically allocates token budgets for audio and video inputs in omni-modal large language models, using skill pools and local complexi…
A First-Principles Derivation of LLM Policy Optimization: From Expected Reward to GRPO and Its Structural Extensions
Jianghan Shen, Siqi Luo, Yue Li +9
Policy gradient algorithms for language models optimize the same objective , which has exactly two factors: the trajectory probability…
Adaptive Optimization for Enhanced Efficiency in Large-Scale Language Model Training
Jiajing Chen, Bingying Liu, Xiaoxuan Liao +3
With the rapid development of natural language processing technology, large-scale language models (LLM) have achieved remarkable results in a variety of tasks. However, how to effe…
The Llama 3 Herd of Models
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri +556
Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models th…