2 papers
cs.CL2024
Understanding the role of FFNs in driving multilingual behaviour in LLMs
Sunit Bhattacharya, Ondřej Bojar
Multilingualism in Large Language Models (LLMs) is an yet under-explored area. In this paper, we conduct an in-depth analysis of the multilingual capabilities of a family of a Larg…
cs.CL2023
Unveiling Multilinguality in Transformer Models: Exploring Language Specificity in Feed-Forward Networks
Sunit Bhattacharya, Ondrej Bojar
Recent research suggests that the feed-forward module within Transformers can be viewed as a collection of key-value memories, where the keys learn to capture specific patterns fro…