Comprehensive Performance Modeling and System Design Insights for Foundation Models

Shashank Subramanian, Ermal Rrapaj, Peter Harrington +6

Generative AI, in particular large transformer models, are increasingly driving HPC system design in science and industry. We analyze performance characteristics of such transforme…