10 papers
Attention-based representations for multi-task computation
Daniel Hsu, Mingyue Xu
Multi-head attention layers produce vector representations that support multiple downstream tasks. We establish bounds on the number of heads required in two simple and concrete mu…
Group-realizable multi-group learning by minimizing empirical risk
Navid Ardeshir, Samuel Deng, Daniel Hsu +1
The sample complexity of multi-group learning is shown to improve in the group-realizable setting over the agnostic setting, even when the family of groups is infinite so long as i…
Time-Aware Synthetic Control
Saeyoung Rho, Cyrus Illick, Samhitha Narasipura +3
The synthetic control (SC) framework is widely used for observational causal inference with time-series panel data. SC has been successful in diverse applications, but existing met…
Panprediction: Optimal Predictions for Any Downstream Task and Loss
Sivaraman Balakrishnan, Nika Haghtalab, Daniel Hsu +2
Supervised learning is classically formulated as training a model to minimize a fixed loss function over a fixed distribution, or task. However, an emerging paradigm instead views…
Prior Knowledge Makes It Possible: From Sublinear Graph Algorithms to LLM Test-Time Methods
Avrim Blum, Daniel Hsu, Cyrus Rashtchian +1
Test-time augmentation, such as Retrieval-Augmented Generation (RAG) or tool use, critically depends on an interplay between a model's parametric knowledge and externally retrieved…
Survey on Algorithms for multi-index models
Joan Bruna, Daniel Hsu
We review the literature on algorithms for estimating the index space in a multi-index model. The primary focus is on computationally efficient (polynomial-time) algorithms in Gaus…
Learning Gaussian Multi-Index Models with Gradient Flow: Time Complexity and Directional Convergence
Berfin Şimşek, Amire Bendjeddou, Daniel Hsu
This work focuses on the gradient flow dynamics of a neural network model that uses correlation loss to approximate a multi-index function on high-dimensional standard Gaussian dat…
Group-wise oracle-efficient algorithms for online multi-group learning
Samuel Deng, Daniel Hsu, Jingwen Liu
We study the problem of online multi-group learning, a learning model in which an online learner must simultaneously achieve small prediction regret on a large collection of (possi…
Multi-group Learning for Hierarchical Groups
Samuel Deng, Daniel Hsu
The multi-group learning model formalizes the learning scenario in which a single predictor must generalize well on multiple, possibly overlapping subgroups of interest. We extend…
Efficient Estimation of the Central Mean Subspace via Smoothed Gradient Outer Products
Gan Yuan, Mingyue Xu, Samory Kpotufe +1
We consider the problem of sufficient dimension reduction (SDR) for multi-index models. The estimators of the central mean subspace in prior works either have slow (non-parametric)…