Apple Neural Engine: Architecture, Programming, and Performance

Abstract

The Apple Neural Engine (ANE) is Apple's fixed-function matrix accelerator. It has shipped in every Apple system-on-chip since the A11 in iPhones and iPads and the M1 in Macs. Applications reach it only through Core ML, which documents almost nothing below its own surface. This work reverse-engineers the engine. We reach it directly, below Core ML, from an ordinary user process, and that access opens the rest to measurement. We measure on Apple silicon and decompile the private runtime, compiler, kernel driver, and firmware, checking each against the other. We give the datapath and the roofline that bounds throughput and energy. Below that are the compiler and the program format it emits, the weight-compression scheme, the kernel driver, the firmware, and the command protocol. We decode compiler targets for the A11 through A18 and M1 through M5 families, with per-chip tables and an operation-by-device matrix. We measured silicon on the M1 and M5 only, and we mark every claim measured, decompile-derived, or predicted. We record how we reached the engine and what we could not measure. The route is callable from user space, but Apple does not document or support it, and it breaks across operating-system updates. It serves measurement and on-device research. Core ML remains the path for shipped software.

arXiv comment303 pages, 12 figures, 218 tables