6 papers
Two Speeds of Learning: A Representation-Readout Decomposition of Grokking and Double Descent
Chi-Ning Chou, Oscar Uzdelewicz, Neng-Chun Chiu +2
Training loss and accuracy are the standard signals used to monitor generalization during deep neural network training. Two well-documented phenomena complicate this picture: in gr…
Linear Readout of Neural Manifolds with Continuous Variables
Will Slatton, Chi-Ning Chou, SueYeon Chung
Brains and artificial neural networks compute with continuous variables such as object position or stimulus orientation. However, the complex variability in neural responses makes…
Diagnosing Generalization Failures from Representational Geometry Markers
Chi-Ning Chou, Artem Kirsanov, Yao-Yuan Yang +1
Generalization, the ability to perform well beyond the training context, is a hallmark of biological and artificial intelligence, yet anticipating unseen failures remains a central…
Feature Learning beyond the Lazy-Rich Dichotomy: Insights from Representational Geometry
Chi-Ning Chou, Hang Le, Yichen Wang +1
Integrating task-relevant information into neural representations is a fundamental ability of both biological and artificial intelligence systems. Recent theories have categorized…
Nonlinear classification of neural manifolds with contextual information
Francesca Mignacco, Chi-Ning Chou, SueYeon Chung
Understanding how neural systems efficiently process information through distributed representations is a fundamental challenge at the interface of neuroscience and machine learnin…
The Geometry of Prompting: Unveiling Distinct Mechanisms of Task Adaptation in Language Models
Artem Kirsanov, Chi-Ning Chou, Kyunghyun Cho +1
Decoder-only language models have the ability to dynamically switch between various computational tasks based on input prompts. Despite many successful applications of prompting, t…