2 citations · 3 across the 7 of their papers we have counts for
7 papers
An Analysis of Residual-Stream Geometry Across Transformer Depth
Sunit Bhattacharya, Ravi Shankar Kolli
We propose a transition-centred geometric analysis of transformer residual streams. Relative displacement measures how \emph{far} representations move between consecutive layers, a…
Understanding the role of FFNs in driving multilingual behaviour in LLMs
Sunit Bhattacharya, Ondřej Bojar
Multilingualism in Large Language Models (LLMs) is an yet under-explored area. In this paper, we conduct an in-depth analysis of the multilingual capabilities of a family of a Larg…
Unveiling Multilinguality in Transformer Models: Exploring Language Specificity in Feed-Forward Networks
Sunit Bhattacharya, Ondrej Bojar
Recent research suggests that the feed-forward module within Transformers can be viewed as a collection of key-value memories, where the keys learn to capture specific patterns fro…
Multimodal Shannon Game with Images
Vilém Zouhar, Sunit Bhattacharya, Ondřej Bojar
The Shannon game has long been used as a thought experiment in linguistics and NLP, asking participants to guess the next letter in a sentence based on its preceding context. We ex…
Sentence Ambiguity, Grammaticality and Complexity Probes
Sunit Bhattacharya, Vilém Zouhar, Ondřej Bojar
It is unclear whether, how and where large pre-trained language models capture subtle linguistic traits like ambiguity, grammaticality and sentence complexity. We present results o…
Team ÚFAL at CMCL 2022 Shared Task: Figuring out the correct recipe for predicting Eye-Tracking features using Pretrained Language Models
Sunit Bhattacharya, Rishu Kumar, Ondrej Bojar
Eye-Tracking data is a very useful source of information to study cognition and especially language comprehension in humans. In this paper, we describe our systems for the CMCL 202…
EMMT: A simultaneous eye-tracking, 4-electrode EEG and audio corpus for multi-modal reading and translation scenarios
Sunit Bhattacharya, Věra Kloudová, Vilém Zouhar +1
We present the Eyetracked Multi-Modal Translation (EMMT) corpus, a dataset containing monocular eye movement recordings, audio and 4-electrode electroencephalogram (EEG) data of 43…