2 papers
cs.LG2025
Constructing Efficient Fact-Storing MLPs for Transformers
Owen Dugan, Roberto Garcia, Ronny Junkins +5
The success of large language models (LLMs) can be attributed in part to their ability to efficiently store factual knowledge as key-value mappings within their MLP parameters. Rec…
cs.LG2025
Towards Learning High-Precision Least Squares Algorithms with Sequence Models
Jerry Liu, Jessica Grogan, Owen Dugan +4
This paper investigates whether sequence models can learn to perform numerical algorithms, e.g. gradient descent, on the fundamental problem of least squares. Our goal is to inheri…