Spilled Energy in Large Language Models

Adrian Robert Minut, Hazem Dewidar, Iacopo Masi

We reinterpret the final Large Language Model (LLM) softmax classifier as an Energy-Based Model (EBM), decomposing the sequence-to-sequence probability chain into multiple interact…