#glu architecture

Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations

Pere Martra, Eugenio Martínez Cámara, Alfonso Ureña López

The paper introduces Fairness Pruning, a method that identifies and zeroes a small set of neurons in GLU-MLP layers of large language models to locate and modulate demographic bias…