5 citations · 22 across the 17 of their papers we have counts for
17 papers
The Llama 4 Herd: Architecture, Training, Evaluation, and Deployment Notes
Redacted by arXiv
This document consolidates publicly reported technical details about Metas Llama 4 model family. It summarizes (i) released variants (Scout and Maverick) and the broader herd conte…
Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformers
Yibo Jiang, Goutham Rajendran, Pradeep Ravikumar +1
Large Language Models (LLMs) have the capacity to store and recall facts. Through experimentation with open-source models, we observe that this ability to retrieve facts can be eas…
Efficient Certificates of Anti-Concentration Beyond Gaussians
Ainesh Bakshi, Pravesh Kothari, Goutham Rajendran +2
A set of high dimensional points in isotropic position is said to be -anti concentrated if for every direction , the fraction of poin…
On the Origins of Linear Representations in Large Language Models
Yibo Jiang, Goutham Rajendran, Pradeep Ravikumar +2
Recent works have argued that high-level semantic concepts are encoded "linearly" in the representation space of large language models. In this work, we study the origins of such l…
Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Models
Goutham Rajendran, Simon Buchholz, Bryon Aragam +2
To build intelligent machine learning systems, there are two broad approaches. One approach is to build inherently interpretable models, as endeavored by the growing field of causa…
An Interventional Perspective on Identifiability in Gaussian LTI Systems with Independent Component Analysis
Goutham Rajendran, Patrik Reizinger, Wieland Brendel +1
We investigate the relationship between system identification and intervention design in dynamical systems. While previous research demonstrated how identifiable representation lea…
Learning Linear Causal Representations from Interventions under General Nonlinear Mixing
Simon Buchholz, Goutham Rajendran, Elan Rosenfeld +3
We study the problem of learning causal representations from unknown, latent interventions in a general setting, where the latent distribution is Gaussian but the mixing function i…
Nonlinear Random Matrices and Applications to the Sum of Squares Hierarchy
Goutham Rajendran
We develop new tools in the theory of nonlinear random matrices and apply them to study the performance of the Sum of Squares (SoS) hierarchy on average-case problems. The SoS hier…
Concentration of polynomial random matrices via Efron-Stein inequalities
Goutham Rajendran, Madhur Tulsiani
Analyzing concentration of large random matrices is a common task in a wide variety of fields. Given independent random variables, many tools are available to analyze random matric…
Combinatorial Optimization via the Sum of Squares Hierarchy
Goutham Rajendran
We study the Sum of Squares (SoS) Hierarchy with a view towards combinatorial optimization. We survey the use of the SoS hierarchy to obtain approximation algorithms on graphs usin…
Analyzing Robustness of End-to-End Neural Models for Automatic Speech Recognition
Goutham Rajendran, Wei Zou
We investigate robustness properties of pre-trained neural models for automatic speech recognition. Real life data in machine learning is usually very noisy and almost never clean,…
Identifiability of deep generative models without auxiliary information
Bohdan Kivva, Goutham Rajendran, Pradeep Ravikumar +1
We prove identifiability of a broad class of deep latent variable models that (a) have universal approximation capabilities and (b) are the decoders of variational autoencoders tha…
Sum-of-Squares Lower Bounds for Sparse Independent Set
Chris Jones, Aaron Potechin, Goutham Rajendran +2
The Sum-of-Squares (SoS) hierarchy of semidefinite programs is a powerful algorithmic paradigm which captures state-of-the-art algorithmic guarantees for a wide array of problems.…
Structure learning in polynomial time: Greedy algorithms, Bregman information, and exponential families
Goutham Rajendran, Bohdan Kivva, Ming Gao +1
Greedy algorithms have long been a workhorse for learning graphical models, and more broadly for learning statistical models with sparse structure. In the context of learning direc…
Learning latent causal graphs via mixture oracles
Bohdan Kivva, Goutham Rajendran, Pradeep Ravikumar +1
We study the problem of reconstructing a causal graphical model from data in the presence of latent variables. The main problem of interest is recovering the causal structure over…
Machinery for Proving Sum-of-Squares Lower Bounds on Certification Problems
Aaron Potechin, Goutham Rajendran
In this paper, we construct general machinery for proving Sum-of-Squares lower bounds on certification problems by generalizing the techniques used by Barak et al. [FOCS 2016] to p…
Sum-of-Squares Lower Bounds for Sherrington-Kirkpatrick via Planted Affine Planes
Mrinalkanti Ghosh, Fernando Granha Jeronimo, Chris Jones +2
The Sum-of-Squares (SoS) hierarchy is a semi-definite programming meta-algorithm that captures state-of-the-art polynomial time guarantees for many optimization problems such as Ma…