1 citations · 1 across the 11 of their papers we have counts for
11 papers
A Formal Limitation on Learning Human Language From Textual Corpora
Emily Cheng, Ryan Cotterell
Can a listener recover what a speaker means from the form of an utterance alone? We answer this question information-theoretically, and for a listener given by any featurizer of te…
Abstraction Induces the Brain Alignment of Language and Speech Models
Emily Cheng, Aditya R. Vaidya, Richard Antonello
Research has repeatedly demonstrated that intermediate hidden states extracted from large language models and speech audio models predict measured brain response to natural languag…
GenCtrl -- A Formal Controllability Toolkit for Generative Models
Emily Cheng, Carmen Amo Alonso, Federico Danieli +4
As generative models become ubiquitous, there is a critical need for fine-grained control over the generation process. Yet, while controlled generation methods from prompting to fi…
Tracing the complexity profiles of different linguistic phenomena through the intrinsic dimension of LLM representations
Marco Baroni, Emily Cheng, Iria de-Dios-Flores +1
We explore intrinsic dimension (ID) of LLM representations as a marker of linguistic complexity. Specifically, we test whether ID differences across model layers reflect well-known…
Principles of semantic and functional efficiency in grammatical patterning
Emily Cheng, Francesca Franzon
Grammatical features such as number and gender serve two central functions in human languages. While they encode salient semantic attributes like numerosity and animacy, they also…
Geometric Signatures of Compositionality Across a Language Model's Lifetime
Jin Hwa Lee, Thomas Jiralerspong, Lei Yu +2
By virtue of linguistic compositionality, few syntactic rules and a finite lexicon can generate an unbounded number of sentences. That is, language, though seemingly high-dimension…
Evidence from fMRI Supports a Two-Phase Abstraction Process in Language Models
Emily Cheng, Richard J. Antonello
Research has repeatedly demonstrated that intermediate hidden states extracted from large language models are able to predict measured brain response to natural language stimuli. Y…
Emergence of a High-Dimensional Abstraction Phase in Language Transformers
Emily Cheng, Diego Doimo, Corentin Kervadec +4
A language model (LM) is a mapping from a linguistic context to an output token. However, much remains to be known about this mapping, including how its geometric properties relate…
LiSeCo: Linear Semantic Control for Language Generation
Emily Cheng, Carmen Amo Alonso
The prevalence of Large Language Models (LLMs) in critical applications highlights the need for controlled language generation methods that are both computationally efficient and e…
Bridging Information-Theoretic and Geometric Compression in Language Models
Emily Cheng, Corentin Kervadec, Marco Baroni
For a language model (LM) to faithfully model human language, it must compress vast, potentially infinite information into relatively few dimensions. We propose analyzing compressi…
On the Correspondence between Compositionality and Imitation in Emergent Neural Communication
Emily Cheng, Mathieu Rita, Thierry Poibeau
Compositionality is a hallmark of human language that not only enables linguistic generalization, but also potentially facilitates acquisition. When simulating language emergence w…