33 citations · 56 across the 24 of their papers we have counts for
25 papers
Spectral Feedback for Test-Time Alignment of Protein Diffusion Models
Shai Dickman, Mert Cemri, Landon Butler +1
Reward maximization alignment methods for discrete diffusion models have primarily focused on steering the reverse process, either by influencing token logits or by selecting favor…
Fantastic Adaptive Taxonomies and How to Use Them
Mert Cemri, Andrei Cojocaru, Melissa Pan +9
An agent system's execution traces record how it fails, and procedures that improve such a system without changing model weights (trajectory selection, prompt and workflow optimiza…
Towards Anytime-Valid Statistical Watermarking
Baihe Huang, Eric Xu, Kannan Ramchandran +2
The proliferation of Large Language Models (LLMs) necessitates efficient mechanisms to distinguish machine-generated content from human text. While statistical watermarking has eme…
Adaptive Sparse Möbius Transforms for Learning Polynomials
Yigit Efe Erginbas, Justin Singh Kang, Elizabeth Polito +1
We consider the problem of exactly learning an -sparse real-valued Boolean polynomial of degree of the form . This problem corresponds t…
A Positive Case for Faithfulness: LLM Self-Explanations Help Predict Model Behavior
Harry Mayne, Justin Singh Kang, Dewi Gould +3
LLM self-explanations are often presented as a promising tool for AI oversight, yet their faithfulness to the model's true reasoning process is poorly understood. Existing faithful…
An Odd Estimator for Shapley Values
Fabian Fumagalli, Landon Butler, Justin Singh Kang +2
The Shapley value is a ubiquitous framework for attribution in machine learning, encompassing feature importance, data valuation, and causal inference. However, its exact computati…
Sample Complexity and Representation Ability of Test-time Scaling Paradigms
Baihe Huang, Shanda Li, Tianhao Wu +5
Test-time scaling paradigms have significantly advanced the capabilities of large language models (LLMs) on complex tasks. Despite their empirical success, theoretical understandin…
: Faster Test-Time Scaling through Speculative Drafts
Mert Cemri, Nived Rajaraman, Rishabh Tiwari +6
Scaling test-time compute has driven the recent advances in the reasoning capabilities of large language models (LLMs), typically by allocating additional computation for more thor…
ProxySPEX: Inference-Efficient Interpretability via Sparse Feature Interactions in LLMs
Landon Butler, Abhineet Agarwal, Justin Singh Kang +3
Large Language Models (LLMs) have achieved remarkable performance by capturing complex interactions between input features. To identify these interactions, most existing approaches…
Double Blind Imaging with Generative Modeling
Brett Levac, Ajil Jalal, Kannan Ramchandran +1
Blind inverse problems in imaging arise from uncertainties in the system used to collect (noisy) measurements of images. Recovering clean images from these measurements typically r…
Online Assortment and Price Optimization Under Contextual Choice Models
Yigit Efe Erginbas, Thomas A. Courtade, Kannan Ramchandran
We consider an assortment selection and pricing problem in which a seller has different items available for sale. In each round, the seller observes a -dimensional contextua…
Why Do Multi-Agent LLM Systems Fail?
Mert Cemri, Melissa Z. Pan, Shuyi Yang +10
Despite enthusiasm for Multi-Agent LLM Systems (MAS), their performance gains on popular benchmarks are often minimal. This gap highlights a critical need for a principled understa…
SPEX: Scaling Feature Interaction Explanations for LLMs
Justin Singh Kang, Landon Butler, Abhineet Agarwal +4
Large language models (LLMs) have revolutionized machine learning due to their ability to capture complex interactions between input features. Popular post-hoc explanation methods…
From Markov to Laplace: How Mamba In-Context Learns Markov Chains
Marco Bondaschi, Nived Rajaraman, Xiuying Wei +5
While transformer-based language models have driven the AI revolution thus far, their computational complexity has spurred growing interest in viable alternatives, such as structur…
EmbedLLM: Learning Compact Representations of Large Language Models
Richard Zhuang, Tianhao Wu, Zhaojin Wen +3
With hundreds of thousands of language models available on Huggingface today, efficiently evaluating and utilizing these models across various downstream, tasks has become increasi…
Transformers on Markov Data: Constant Depth Suffices
Nived Rajaraman, Marco Bondaschi, Kannan Ramchandran +2
Attention-based transformers have been remarkably successful at modeling generative processes across various domains and modalities. In this paper, we study the behavior of transfo…
Toward a Theory of Tokenization in LLMs
Nived Rajaraman, Jiantao Jiao, Kannan Ramchandran
While there has been a large body of research attempting to circumvent tokenization for language modeling (Clark et al., 2022; Xue et al., 2022), the current consensus is that it i…
Pairwise Proximal Policy Optimization: Harnessing Relative Feedback for LLM Alignment
Tianhao Wu, Banghua Zhu, Ruoyu Zhang +3
Large Language Models (LLMs) can acquire extensive world knowledge through pre-training on large corpora. However, due to exposure to low-quality data, LLMs may exhibit harmful beh…
Greedy Pruning with Group Lasso Provably Generalizes for Matrix Sensing
Nived Rajaraman, Devvrit, Aryan Mokhtari +1
Pruning schemes have been widely used in practice to reduce the complexity of trained models with a massive number of parameters. In fact, several practical studies have shown that…
Statistical Complexity and Optimal Algorithms for Non-linear Ridge Bandits
Nived Rajaraman, Yanjun Han, Jiantao Jiao +1
We consider the sequential decision-making problem where the mean outcome is a non-linear function of the chosen action. Compared with the linear model, two curious phenomena arise…
Efficiently Computing Sparse Fourier Transforms of -ary Functions
Yigit Efe Erginbas, Justin Singh Kang, Amirali Aghazadeh +1
Fourier transformations of pseudo-Boolean functions are popular tools for analyzing functions of binary sequences. Real-world functions often have structures that manifest in a spa…
Spectral Regularization Allows Data-frugal Learning over Combinatorial Spaces
Amirali Aghazadeh, Nived Rajaraman, Tony Tu +1
Data-driven machine learning models are being increasingly employed in several important inference problems in biology, chemistry, and physics which require learning over combinato…
Neurotoxin: Durable Backdoors in Federated Learning
Zhengming Zhang, Ashwinee Panda, Linyue Song +5
Due to their decentralized nature, federated learning (FL) systems have an inherent vulnerability during their training to adversarial backdoor attacks. In this type of attack, the…
Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data
Yaoqing Yang, Ryan Theisen, Liam Hodgkinson +4
Selecting suitable architecture parameters and training hyperparameters is essential for enhancing machine learning (ML) model performance. Several recent empirical studies conduct…