7 papers
Hyperloop Transformers
Abbas Zeitoun, Lucas Torroba-Hennigen, Yoon Kim
LLM architecture research generally aims to maximize model quality subject to fixed compute/latency budgets. However, many applications of interest such as edge and on-device deplo…
ELF: Embedded Language Flows
Keya Hu, Linlu Qiu, Yiyang Lu +5
Diffusion and flow-based models have become the de facto approaches for generating continuous data, e.g., in domains such as images and videos. Their success has attracted growing…
Variable-Width Transformers
Zhaofeng Wu, Oliver Sieberling, Shawn Tan +3
Scaling model size, specifically depth and width, has driven significant progress in transformer-based language models. However, most architectures maintain a constant width across…
Dynamic Short Convolutions Improve Transformers
Oliver Sieberling, Bharat Runwal, Rameswar Panda +1
Transformers have become the dominant architecture for large language models, largely due to the scalability and flexibility of attention, feed-forward layers, residual connections…
CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs
Han Guo, Jack Zhang, Arjun Menon +4
Transformer training systems are built around dense linear algebra, yet a nontrivial fraction of end-to-end time is spent on surrounding memory-bound operators. Normalization, acti…
Ladder-residual: parallelism-aware architecture for accelerating large model inference with communication overlapping
Muru Zhang, Mayank Mishra, Zhongzhu Zhou +7
Large language model inference is both memory-intensive and time-consuming, often requiring distributed algorithms to efficiently scale. Various model parallelism strategies are us…