3 papers
cs.CL2024
Probing Context Localization of Polysemous Words in Pre-trained Language Model Sub-Layers
Soniya Vijayakumar, Josef van Genabith, Simon Ostermann
In the era of high performing Large Language Models, researchers have widely acknowledged that contextual word representations are one of the key drivers in achieving top performan…
cs.CL2023
Where exactly does contextualization in a PLM happen?
Soniya Vijayakumar, Tanja Bäumel, Simon Ostermann +1
Pre-trained Language Models (PLMs) have shown to be consistently successful in a plethora of NLP tasks due to their ability to learn contextualized representations of words (Ethaya…
cs.CL2023
Investigating the Encoding of Words in BERT's Neurons using Feature Textualization
Tanja Baeumel, Soniya Vijayakumar, Josef van Genabith +2
Pretrained language models (PLMs) form the basis of most state-of-the-art NLP technologies. Nevertheless, they are essentially black boxes: Humans do not have a clear understanding…