INT-FP-QSim: Mixed Precision and Formats For Large Language Models and Vision Transformers

Lakshmi Nair, Mikhail Bernadskiy, Arulselvan Madhavan +3

The recent rise of large language models (LLMs) has resulted in increased efforts towards running LLMs at reduced precision. Running LLMs at lower precision supports resource const…