3 citations · 3 across the 7 of their papers we have counts for
9 papers
GAPS: Dimension-Level Gates for Conditional Activation Steering
Moghis Fereidouni, Muhammad Umair Haider, Hassan Sajjad +1
Activation steering suppresses undesired behaviors in language models by adding a steering vector to the hidden state during generation. Recent conditional methods such as CAST and…
On the Persistent Effects of Lexicality in Large Language Models
Hammad Rizwan, Muhammad Umair Haider, Nishant Subramani +3
Representations extracted from large language models (LLMs) play an important role in many downstream applications. However, the structure of these representations is often influen…
Multi-Granular Node Pruning for Causal Circuit Discovery
Muhammad Umair Haider, Hammad Rizwan, Hassan Sajjad +1
Circuit discovery aims to identify minimal subnetworks that are responsible for specific behaviors in large language models (LLMs). Existing approaches primarily rely on iterative…
Evaluating Sparse Autoencoders for Monosemantic Representation
Moghis Fereidouni, Muhammad Umair Haider, Peizhong Ju +1
A key barrier to interpreting large language models is polysemanticity, where neurons activate for multiple unrelated concepts. Sparse autoencoders (SAEs) have been proposed to mit…
Neurons Speak in Ranges: Breaking Free from Discrete Neuronal Attribution
Muhammad Umair Haider, Hammad Rizwan, Hassan Sajjad +2
Pervasive polysemanticity in large language models (LLMs) undermines discrete neuron-concept attribution, posing a significant challenge for model interpretation and control. We sy…
Looking into Black Box Code Language Models
Muhammad Umair Haider, Umar Farooq, A. B. Siddique +1
Language Models (LMs) have shown their application for tasks pertinent to code and several code~LMs have been proposed recently. The majority of the studies in this direction only…
Neural Network Pruning Through Constrained Reinforcement Learning
Shehryar Malik, Muhammad Umair Haider, Omer Iqbal +1
Network pruning reduces the size of neural networks by removing (pruning) neurons such that the performance drop is minimal. Traditional pruning approaches focus on designing metri…
Comprehensive Online Network Pruning via Learnable Scaling Factors
Muhammad Umair Haider, Murtaza Taj
One of the major challenges in deploying deep neural network architectures is their size which has an adverse effect on their inference time and memory requirements. Deep CNNs can…
Teacher-Class Network: A Neural Network Compression Mechanism
Shaiq Munir Malik, Muhammad Umair Haider, Mohbat Tharani +2
To reduce the overwhelming size of Deep Neural Networks (DNN) teacher-student methodology tries to transfer knowledge from a complex teacher network to a simple student network. We…