Inference Optimizations for Large Language Models: Effects, Challenges, and Practical Considerations

Leo Donisch, Sigurd Schacht, Carsten Lanquillon

Large language models are ubiquitous in natural language processing because they can adapt to new tasks without retraining. However, their sheer scale and complexity present unique…