Towards Explainability of SLMs by investigating Token Level Activation

Sayantani Ghosh, Rajashik Datta, Amit Kumar Das +1

Transformer-based language models such as BERT having 110M+ parameters have revolutionized natural language understanding, yet their internal mechanisms remain largely opaque to re…