2 papers
cs.CL2024
How do languages influence each other? Studying cross-lingual data sharing during LM fine-tuning
Rochelle Choenni, Dan Garrette, Ekaterina Shutova
Multilingual large language models (MLLMs) are jointly trained on data from many different languages such that representation of individual languages can benefit from other languag…
cs.CL2024
The Impact of Depth on Compositional Generalization in Transformer Language Models
Jackson Petty, Sjoerd van Steenkiste, Ishita Dasgupta +3
To process novel sentences, language models (LMs) must generalize compositionally -- combine familiar elements in new ways. What aspects of a model's structure promote compositiona…