6 papers
Constructing Efficient Fact-Storing MLPs for Transformers
Owen Dugan, Roberto Garcia, Ronny Junkins +5
The success of large language models (LLMs) can be attributed in part to their ability to efficiently store factual knowledge as key-value mappings within their MLP parameters. Rec…
Operationalizing Justice: Towards the Development of a Principle Based Design Framework for Human Services AI
Maria Y. Rodriguez, Seventy Hall, Pranav Sankhe +4
Scholars investigating ethical AI, especially in high stakes settings like child welfare, have arguably been seeking ways to embed notions of justice into the design of these criti…
Identifying Subgroup and Context Effects in Conjoint Experiments
Steven Wang, Isys Johnson, Jessica Grogan +4
Conjoint experiments have become central to survey research in political science and related fields because they allow researchers to study preferences across multiple attributes s…
BWLer: Barycentric Weight Layer Elucidates a Precision-Conditioning Tradeoff for PINNs
Jerry Liu, Yasa Baig, Denise Hui Jean Lee +3
Physics-informed neural networks (PINNs) offer a flexible way to solve partial differential equations (PDEs) with machine learning, yet they still fall well short of the machine-pr…
Cartridges: Lightweight and general-purpose long context representations via self-study
Sabri Eyuboglu, Ryan Ehrlich, Simran Arora +8
Large language models are often used to answer queries grounded in large text corpora (e.g. codebases, legal documents, or chat histories) by placing the entire corpus in the conte…
Towards Learning High-Precision Least Squares Algorithms with Sequence Models
Jerry Liu, Jessica Grogan, Owen Dugan +4
This paper investigates whether sequence models can learn to perform numerical algorithms, e.g. gradient descent, on the fundamental problem of least squares. Our goal is to inheri…