565 citations · 1.1k across the 57 of their papers we have counts for
13 papers · 1 filter
SMAC-Hard: Enabling Mixed Opponent Strategy Script and Self-play on SMAC
Yue Deng, Yan Yu, Weiyu Ma +4
The availability of challenging simulation environments is pivotal for advancing the field of Multi-Agent Reinforcement Learning (MARL). In cooperative MARL settings, the StarCraft…
HiMemFormer: Hierarchical Memory-Aware Transformer for Multi-Agent Action Anticipation
Zirui Wang, Xinran Zhao, Simon Stepputtis +4
Understanding and predicting human actions has been a long-standing challenge and is a crucial measure of perception in robotics AI. While significant progress has been made in ant…
Learning Humanoid Locomotion with Perceptive Internal Model
Junfeng Long, Junli Ren, Moji Shi +4
In contrast to quadruped robots that can navigate diverse terrains using a "blind" policy, humanoid robots require accurate perception for stable locomotion due to their high degre…
MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuning
Haotian Zhang, Mingfei Gao, Zhe Gan +20
We present MM1.5, a new family of multimodal large language models (MLLMs) designed to enhance capabilities in text-rich image understanding, visual referring and grounding, and mu…
MMAU: A Holistic Benchmark of Agent Capabilities Across Diverse Domains
Guoli Yin, Haoping Bai, Shuang Ma +21
Recent advances in large language models (LLMs) have increased the demand for comprehensive benchmarks to evaluate their capabilities as human-like agents. Existing benchmarks, whi…
ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities
Jiarui Lu, Thomas Holleis, Yizhe Zhang +9
Recent large language models (LLMs) advancements sparked a growing research interest in tool assisted LLMs solving real-world challenges, which calls for comprehensive evaluation o…
GRUtopia: Dream General Robots in a City at Scale
Hanqing Wang, Jiahe Chen, Wensi Huang +19
Recent works have been exploring the scaling laws in the field of Embodied AI. Given the prohibitive costs of collecting real-world data, we believe the Simulation-to-Real (Sim2Rea…
Understanding Alignment in Multimodal LLMs: A Comprehensive Study
Elmira Amirloo, Jean-Philippe Fauconnier, Christoph Roesmann +8
Preference alignment has become a crucial component in enhancing the performance of Large Language Models (LLMs), yet its impact in Multimodal Large Language Models (MLLMs) remains…
Apple Intelligence Foundation Language Models
Tom Gunter, Zirui Wang, Chong Wang +152
We present foundation language models developed to power Apple Intelligence features, including a ~3 billion parameter model designed to run efficiently on devices and a large serv…
Revisiting MoE and Dense Speed-Accuracy Comparisons for LLM Training
Xianzhi Du, Tom Gunter, Xiang Kong +5
Mixture-of-Experts (MoE) enjoys performance gain by increasing model capacity while keeping computation cost constant. When comparing MoE to dense models, prior work typically adop…
MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training
Brandon McKinzie, Zhe Gan, Jean-Philippe Fauconnier +29
In this work, we discuss building performant Multimodal Large Language Models (MLLMs). In particular, we study the importance of various architecture components and data choices. T…
Learning H-Infinity Locomotion Control
Junfeng Long, Wenye Yu, Quanyi Li +3
Stable locomotion in precipitous environments is an essential task for quadruped robots, requiring the ability to resist various external disturbances. Recent neural policies enhan…
Hybrid Internal Model: Learning Agile Legged Locomotion with Simulated Robot Response
Junfeng Long, Zirui Wang, Quanyi Li +3
Robust locomotion control depends on accurate state estimations. However, the sensors of most legged robots can only provide partial and noisy observations, making the estimation p…