1 citations · 2 across the 10 of their papers we have counts for
10 papers
How Data Shapes RoPE Frequency Usage: From Positional Scale Matching to Length Generalization
Xinyi Wu, Siyuan Liu, Ali Jadbabaie
Rotary Position Embeddings (RoPE) provide transformers with a fixed grid of positional frequencies, yet trained models use these frequencies highly non-uniformly. We study what det…
Is Your Conditional Diffusion Model Actually Denoising?
Daniel Pfrommer, Zehao Dou, Christopher Scarvelis +2
We study the inductive biases of diffusion models with a conditioning-variable, which have seen widespread application as both text-conditioned generative image models and observat…
Unbiased Regression-Adjusted Estimation of Average Treatment Effects in Randomized Controlled Trials
Alberto Abadie, Mehrdad Ghadiri, Ali Jadbabaie +1
This article introduces a leave-one-out regression adjustment (LOORA) for estimating average treatment effects in randomized controlled trials. In finite samples, LOORA removes the…
A Test-Function Approach to Incremental Stability
Daniel Pfrommer, Max Simchowitz, Ali Jadbabaie
This paper presents a novel framework for analyzing Incremental-Input-to-State Stability (ISS) based on the idea of using rewards as "test functions." Whereas control theory tra…
The Pitfalls of Imitation Learning when Actions are Continuous
Max Simchowitz, Daniel Pfrommer, Ali Jadbabaie
We study the problem of imitating an expert demonstrator in a discrete-time, continuous state-and-action control system. We show that, even if the dynamics satisfy a control-theore…
Fast Tensor Completion via Approximate Richardson Iteration
Mehrdad Ghadiri, Matthew Fahrbach, Yunbum Kook +1
We study tensor completion (TC) through the lens of low-rank tensor decomposition (TD). Many TD algorithms use fast alternating minimization methods to solve highly structured line…
On the Emergence of Position Bias in Transformers
Xinyi Wu, Yifei Wang, Stefanie Jegelka +1
Recent studies have revealed various manifestations of position bias in transformer architectures, from the "lost-in-the-middle" phenomenon to attention sinks, yet a comprehensive…
GraphHash: Graph Clustering Enables Parameter Efficiency in Recommender Systems
Xinyi Wu, Donald Loveland, Runjin Chen +7
Deep recommender systems rely heavily on large embedding tables to handle high-cardinality categorical features such as user/item identifiers, and face significant memory constrain…
Improved Sample Complexity of Imitation Learning for Barrier Model Predictive Control
Daniel Pfrommer, Swati Padmanabhan, Kwangjun Ahn +4
Recent work in imitation learning has shown that having an expert controller that is both suitably smooth and stable enables stronger guarantees on the performance of the learned c…
Linear attention is (maybe) all you need (to understand transformer optimization)
Kwangjun Ahn, Xiang Cheng, Minhak Song +3
Transformer training is notoriously difficult, requiring a careful design of optimizers and use of various heuristics. We make progress towards understanding the subtleties of trai…