Direction is what you need: Improving Word Embedding Compression in Large Language Models

Klaudia Bałazy, Mohammadreza Banaei, Rémi Lebret +2

The adoption of Transformer-based models in natural language processing (NLP) has led to great success using a massive number of parameters. However, due to deployment constraints…