most citedOn the Origins of Linear Representations in Large Language Models

2 citations · 4 across the 6 of their papers we have counts for

collaborators

6 papers

cs.LG2025

Taylor-Model Physics-Informed Neural Networks (PINNs) for Ordinary Differential Equations

Chandra Kanth Nagesh, Sriram Sankaranarayanan, Ramneet Kaur +2

We study the problem of learning neural network models for Ordinary Differential Equations (ODEs) with parametric uncertainties. Such neural network models capture the solution to…

stat.ML2024

Markov Equivalence and Consistency in Differentiable Structure Learning

Chang Deng, Kevin Bello, Pradeep Ravikumar +1

Existing approaches to differentiable structure learning of directed acyclic graphs (DAGs) rely on strong identifiability assumptions in order to guarantee that global minimizers o…

cs.LG2024

Identifying General Mechanism Shifts in Linear Causal Representations

Tianyu Chen, Kevin Bello, Francesco Locatello +2

We consider the linear causal representation learning setting where we observe a linear mixing of unknown latent factors, which follow a linear structural causal model. Recent…

cs.CL2024

Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformers

Yibo Jiang, Goutham Rajendran, Pradeep Ravikumar +1

Large Language Models (LLMs) have the capacity to store and recall facts. Through experimentation with open-source models, we observe that this ability to retrieve facts can be eas…

cs.CL2024★ 2 cited

On the Origins of Linear Representations in Large Language Models

Yibo Jiang, Goutham Rajendran, Pradeep Ravikumar +2

Recent works have argued that high-level semantic concepts are encoded "linearly" in the representation space of large language models. In this work, we study the origins of such l…

cs.LG2024★ 2 cited

Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Models

Goutham Rajendran, Simon Buchholz, Bryon Aragam +2

To build intelligent machine learning systems, there are two broad approaches. One approach is to build inherently interpretable models, as endeavored by the growing field of causa…