Eigenpruning: an Interpretability-Inspired PEFT Method

Tomás Vergara-Browne, Álvaro Soto, Akiko Aizawa

We introduce eigenpruning, a method that removes singular values from weight matrices in an LLM to improve its performance in a particular task. This method is inspired by interpre…