11 papers
Apple Intelligence Foundation Language Models
Tom Gunter, Zirui Wang, Chong Wang +152
We present foundation language models developed to power Apple Intelligence features, including a ~3 billion parameter model designed to run efficiently on devices and a large serv…
AXLearn: Modular, Hardware-Agnostic Large Model Training
Mark Lee, Chang Lan, Tom Gunter +34
AXLearn is a production system which facilitates scalable and high-performance training of large deep learning models. Compared to other state-of-art deep learning systems, AXLearn…
Parallel Track Transformers: Enabling Fast GPU Inference with Reduced Synchronization
Chong Wang, Nan Du, Tom Gunter +8
Efficient large-scale inference of transformer-based large language models (LLMs) remains a fundamental systems challenge, frequently requiring multi-GPU parallelism to meet string…
Task-Aware Multi-Expert Architecture For Lifelong Deep Learning
Jianyu Wang, Jacob Nean-Hua Sheikh, Cat P. Le +1
Lifelong deep learning (LDL) trains neural networks to learn sequentially across tasks while preserving prior knowledge. We propose Task-Aware Multi-Expert (TAME), a continual lear…
RLAX: Large-Scale, Distributed Reinforcement Learning for Large Language Models on TPUs
Runlong Zhou, Lefan Zhang, Shang-Chen Wu +29
Reinforcement learning (RL) has emerged as the de-facto paradigm for improving the reasoning capabilities of large language models (LLMs). We have developed RLAX, a scalable RL fra…
Apple Intelligence Foundation Language Models: Tech Report 2025
Ethan Li, Anders Boesen Lindbo Larsen, Chen Zhang +395
We introduce two multilingual, multimodal foundation language models that power Apple Intelligence features across Apple devices and services: i a 3B-parameter on-device model opti…