5 papers
Just-in-time and distributed task representations in language models
Yuxuan Li, Declan Campbell, Stephanie C. Y. Chan +1
Many of language models' impressive capabilities originate from their in-context learning: based on instructions or examples, they can infer and perform new tasks without weight up…
On the generalization of language models from in-context learning and finetuning: a controlled study
Andrew K. Lampinen, Arslan Chaudhry, Stephanie C. Y. Chan +7
Large language models exhibit exciting capabilities, yet can show surprisingly narrow generalization from finetuning. E.g. they can fail to generalize to simple reversals of relati…
Representation biases: will we achieve complete understanding by analyzing representations?
Andrew Kyle Lampinen, Stephanie C. Y. Chan, Yuxuan Li +1
A common approach in neuroscience is to study neural representations as a means to understand a system -- increasingly, by relating the neural representations to the internal repre…
The broader spectrum of in-context learning
Andrew Kyle Lampinen, Stephanie C. Y. Chan, Aaditya K. Singh +1
The ability of language models to learn a task from a few examples in context has generated substantial interest. Here, we provide a perspective that situates this type of supervis…
Learned feature representations are biased by complexity, learning order, position, and more
Andrew Kyle Lampinen, Stephanie C. Y. Chan, Katherine Hermann
Representation learning, and interpreting learned representations, are key areas of focus in machine learning and neuroscience. Both fields generally use representations as a means…