41 citations · 143 across the 39 of their papers we have counts for
41 papers
Adaptive Mean Estimation by In-Context Learning: A Gradient-Flow Analysis
Martin Eppert, Krishna Balasubramanian, Subhro Ghosh +2
Prior Fitted Networks (PFNs) such as TabPFN now rival established statistical procedures across prediction and estimation tasks. A natural explanation is that PFNs have the propert…
Statistical attribute alignment for black-box generative AI via output post-processing
Kevin Jiang, Morgane Austern, Edgar Dobriban +1
Generative AI systems are increasingly used, but aligning their outputs with user requirements poses a continuing challenge. Here, we aim to ensure that the distribution of an attr…
High-Dimensional Statistics: Reflections on Progress and Open Problems
Arian Maleki, Subhabrata Sen, Sivaraman Balakrishnan +9
Over the past two decades, the field of high-dimensional statistics has experienced substantial progress, driven largely by technological advances that have dramatically reduced th…
Coupled Training with Privileged Information and Unlabeled Data
Jiahao Shi, Omar Hagrass, Jason M. Klusowski
In many prediction problems, we have extra information during training (for example, measurements that are expensive or slow to collect) that will not be available when the model i…
Classification Imbalance as Transfer Learning
Eric Xia, Jason M. Klusowski
Classification imbalance arises when one class is much rarer than the other. We frame this setting as transfer learning under label (prior) shift between an imbalanced source distr…
Accuracy Limits of Causal Trees for Individualized Treatment Effects
Matias D. Cattaneo, Jason M. Klusowski, Ruiqi Rae Yu
Recursive decision trees are widely used to estimate heterogeneous causal treatment effects in experimental and observational studies. These methods are typically implemented using…
Sharp Trade-Offs in High-Dimensional Inference via 2-Level SLOPE
Zhiqi Bu, Jason M. Klusowski, Cynthia Rush +1
Among techniques for high-dimensional linear regression, Sorted L-One Penalized Estimation (SLOPE) generalizes the LASSO via an adaptive regularization that applies heavier p…
Revisiting Randomization in Greedy Model Search
Xin Chen, Jason M. Klusowski, Yan Shuo Tan +1
Feature subsampling is a core component of random forests and other ensemble methods. While recent theory suggests that this randomization acts solely as a variance reduction mecha…
One-Layer Transformer Provably Learns One-Nearest Neighbor In Context
Zihao Li, Yuan Cao, Cheng Gao +5
Transformers have achieved great success in recent years. Interestingly, transformers have shown particularly strong in-context learning capability -- even without fine-tuning, the…
Statistical-Computational Trade-offs for Recursive Adaptive Partitioning Estimators
Yan Shuo Tan, Jason M. Klusowski, Krishnakumar Balasubramanian
Models based on recursive adaptive partitioning such as decision trees and their ensembles are popular for high-dimensional regression as they can potentially avoid the curse of di…
Global Convergence in Training Large-Scale Transformers
Cheng Gao, Yuan Cao, Zihao Li +5
Despite the widespread success of Transformers across various domains, their optimization guarantees in large-scale model settings are not well-understood. This paper rigorously an…
Decoding Game: On Minimax Optimality of Heuristic Text Generation Strategies
Sijin Chen, Omar Hagrass, Jason M. Klusowski
Decoding strategies play a pivotal role in text generation for modern language models, yet a puzzling gap divides theory and practice. Surprisingly, strategies that should intuitiv…
Challenges in Variable Importance Ranking Under Correlation
Annie Liang, Thomas Jemielita, Andy Liaw +4
Variable importance plays a pivotal role in interpretable machine learning as it helps measure the impact of factors on the output of the prediction model. Model agnostic methods b…
Stochastic Gradient Descent for Nonparametric Additive Regression
Xin Chen, Jason M. Klusowski
This paper introduces an iterative algorithm for training nonparametric additive models that enjoys favorable memory storage and computational requirements. The algorithm can be vi…
Inference with Mondrian Random Forests
Matias D. Cattaneo, Jason M. Klusowski, William G. Underwood
Random forests are popular methods for regression and classification analysis, and many different variants have been proposed in recent years. One interesting example is the Mondri…
Robust Transfer Learning with Unreliable Source Data
Jianqing Fan, Cheng Gao, Jason M. Klusowski
This paper addresses challenges in robust transfer learning stemming from ambiguity in Bayes classifiers and weak transferable signals between the target and source distribution. W…
Error Reduction from Stacked Regressions
Xin Chen, Jason M. Klusowski, Yan Shuo Tan
Stacking regressions is an ensemble technique that forms linear combinations of different regression estimators to enhance predictive accuracy. The conventional approach uses cross…
On the Implicit Bias of Adam
Matias D. Cattaneo, Jason M. Klusowski, Boris Shigida
In previous literature, backward error analysis was used to find ordinary differential equations (ODEs) approximating the gradient descent trajectory. It was found that finite step…
Sharp Convergence Rates for Matching Pursuit
Jason M. Klusowski, Jonathan W. Siegel
We study the fundamental limits of matching pursuit, or the pure greedy algorithm, for approximating a target function by a linear combination of elements from a di…
On the Pointwise Behavior of Recursive Partitioning and Its Implications for Heterogeneous Causal Effect Estimation
Matias D. Cattaneo, Jason M. Klusowski, Peter M. Tian
Decision tree learning is increasingly being used for pointwise inference. Important applications include causal heterogenous treatment effects and dynamic policy decisions, as wel…
Convergence Rates of Oblique Regression Trees for Flexible Function Libraries
Matias D. Cattaneo, Rajita Chandak, Jason M. Klusowski
We develop a theoretical framework for the analysis of oblique decision trees, where the splits at each decision node occur at linear combinations of the covariates (as opposed to…
Characterizing the SLOPE Trade-off: A Variational Perspective and the Donoho-Tanner Limit
Zhiqi Bu, Jason Klusowski, Cynthia Rush +1
Sorted l1 regularization has been incorporated into many methods for solving high-dimensional statistical estimation problems, including the SLOPE estimator in linear regression. I…
Large Scale Prediction with Decision Trees
Jason M. Klusowski, Peter M. Tian
This paper shows that decision trees constructed with Classification and Regression Trees (CART) and C4.5 methodology are consistent for regression and classification tasks, even w…
Nonparametric Variable Screening with Optimal Decision Stumps
Jason M. Klusowski, Peter M. Tian
Decision trees and their ensembles are endowed with a rich set of diagnostic tools for ranking and screening variables in a predictive model. Despite the widespread use of tree bas…