3 papers
cs.LG2025
Distinct Computations Emerge From Compositional Curricula in In-Context Learning
Jin Hwa Lee, Andrew K. Lampinen, Aaditya K. Singh +1
In-context learning (ICL) research often considers learning a function in-context through a uniform sample of input-output pairs. Here, we investigate how presenting a compositiona…
q-bio.NC2024
Range, not Independence, Drives Modularity in Biologically Inspired Representations
Will Dorrell, Kyle Hsu, Luke Hollingsworth +6
Why do biological and artificial neurons sometimes modularise, each encoding a single meaningful variable, and sometimes entangle their representation of many variables? In this wo…
cs.CL2024
Geometric Signatures of Compositionality Across a Language Model's Lifetime
Jin Hwa Lee, Thomas Jiralerspong, Lei Yu +2
By virtue of linguistic compositionality, few syntactic rules and a finite lexicon can generate an unbounded number of sentences. That is, language, though seemingly high-dimension…