LLM Architecture, Scaling Laws, and Economics: A Quick Summary

William H. Press

The current standard architecture of Large Language Models (LLMs) with QKV self-attention is briefly summarized, including the architecture of a typical Transformer. Scaling laws f…