1 citations · 1 across the 18 of their papers we have counts for
23 papers
Transformers as Adaptive Kernel Learners across Tasks for In-Context Learning
Zhaiming Shen, Alexander Cloninger, Rongjie Lai +1
Transformers have emerged as a dominant architecture in modern machine learning, with in-context learning (ICL) enabling models to adapt to new tasks by conditioning on prompts, wi…
Residual-Stream Burden Shapes Representation Learning in Diffusion Transformers
Tongtong Liang, Siqi Kou, Ziqiao Xi +6
In diffusion-based generation, a neural network can be trained to predict the clean data, the noise, or the velocity from a noisy input. These prediction targets are interconvertib…
Transformers as Cross-Task Learners: Shared Structure Drives Sample Efficiency in In-Context Learning
Zhongjie Shi, Rongjie Lai, Alexander Cloninger +1
Transformers achieve remarkable performance by jointly learning broad families of tasks during pretraining and adapting to unseen tasks from only a short prompt. Yet a rigorous mat…
Riemannian ascent--descent for nonconvex nonconcave minimax landscapes: convergence to basin saddle points and applications to distributionally robust optimization
Rishabh Dixit, Pranav Upadrashta, Alex Cloninger
We study a class of distributionally robust optimization (DRO) problems for the statistical risk problem, formulated as minimax problems over the product of a Euclidean space and a…
Scalable Graph Coreset Selection via Greedy Sampling
Zhaiming Shen, Alexander Cloninger
Sampling representative nodes from large graphs is fundamental to graph signal processing and network analysis, yet existing methods require access to the full graph Laplacian, mak…
Unbalanced Optimal Transport Dictionary Learning for Unsupervised Hyperspectral Image Clustering
Joshua Lentz, Nicholas Karris, Alex Cloninger +1
Hyperspectral images capture vast amounts of high-dimensional spectral information about a scene, making labeling an intensive task that is resistant to out-of-the-box statistical…
Does Sparse Connectivity Improve Generalization? Convolutional Networks Below the Edge of Stability
Tongtong Liang, Esha Singh, Rahul Parhi +2
Gradient descent on overparameterized neural networks typically operates at the Edge of Stability (EoS), where the largest Hessian eigenvalue hovers around a step-size-dependent th…
Optimal Transport, Timesteppers, Newton-Krylov Methods and Steady States of Collective Particle Dynamics
Hannes Vandecasteele, Nicholas Karris, Alexander Cloninger +1
Timesteppers constitute a powerful tool in modern computational science and engineering. Although they are typically used to advance the system forward in time, they can also be vi…
Linearized Optimal Transport for Analysis of High-Dimensional Point-Cloud and Single-Cell Data
Tianxiang Wang, Yingtong Ke, Dhananjay Bhaskar +2
Single-cell technologies generate high-dimensional point clouds of cells, enabling detailed characterization of complex patient states and treatment responses. Yet each patient is…
Revisiting Meta-Learning with Noisy Labels: Reweighting Dynamics and Theoretical Guarantees
Yiming Zhang, Chester Holtz, Gal Mishne +1
Learning with noisy labels remains challenging because over-parameterized networks memorize corrupted supervision. Meta-learning-based sample reweighting mitigates this by using a…
Generalization Below the Edge of Stability: The Role of Data Geometry
Tongtong Liang, Alexander Cloninger, Rahul Parhi +1
Understanding generalization in overparameterized neural networks hinges on the interplay between the data geometry, neural architecture, and training dynamics. In this paper, we t…
Robust Tangent Space Estimation via Laplacian Eigenvector Gradient Orthogonalization
Dhruv Kohli, Sawyer J. Robertson, Gal Mishne +1
Estimating the tangent spaces of a data manifold is a fundamental problem in geometric data analysis. The standard approach, Local Principal Component Analysis (LPCA), struggles in…
KAIROS: Scalable Model-Agnostic Data Valuation
Jiongli Zhu, Parjanya Prajakta Prashant, Alex Cloninger +1
Training data increasingly shapes not only model accuracy but also regulatory compliance and market valuation of AI assets. Yet existing valuation methods remain inadequate: model-…
Transformers for Learning on Noisy and Task-Level Manifolds: Approximation and Generalization Insights
Zhaiming Shen, Alex Havrilla, Rongjie Lai +2
Transformers serve as the foundational architecture for large language and video generation models, such as GPT, BERT, SORA and their successors. Empirical studies have demonstrate…
Using Causal Inference to Explore Government Policy Impact on Computer Usage
Mingjia Zhu, Lechuan Wang, Julien Sebot +3
We explore the causal relationship between COVID-19 lockdown policies and changes in personal computer usage. In particular, we examine how lockdown policies affected average daily…
Robust Graph-Based Semi-Supervised Learning via -Conductances
Sawyer Jack Robertson, Chester Holtz, Zhengchao Wan +2
We study the problem of semi-supervised learning on graphs in the regime where data labels are scarce or possibly corrupted. We propose an approach called -conductance learning…
Linearized Optimal Transport pyLOT Library: A Toolkit for Machine Learning on Point Clouds
Jun Linwu, Varun Khurana, Nicholas Karris +1
The pyLOT library offers a Python implementation of linearized optimal transport (LOT) techniques and methods to use in downstream tasks. The pipeline embeds probability distributi…
Robust boundary detection and density estimation using doubly stochastic scaling of the Gaussian kernel
Dhruv Kohli, Jesse He, Chester Holtz +2
This paper addresses the problem of detecting boundary points and estimating the sampling density of a dataset derived from a compact manifold with boundary, potentially in the pre…
Using Linearized Optimal Transport to Predict the Evolution of Stochastic Particle Systems
Nicholas Karris, Evangelos A. Nikitopoulos, Ioannis G. Kevrekidis +2
We develop an Euler-type method to predict the evolution of a time-dependent probability measure without explicitly learning an operator that governs its evolution. We use lineariz…
Resistance Distance and Linearized Optimal Transport on Graphs
Sawyer Robertson, Zhengchao Wan, Alexander Cloninger
We study the linearization of a discrete transportation distance between probability distributions on finite weighted graphs originally due to Maas (``Gradient flows of the entropy…
OTClean: Data Cleaning for Conditional Independence Violations using Optimal Transport
Alireza Pirhadi, Mohammad Hossein Moslemi, Alexander Cloninger +2
Ensuring Conditional Independence (CI) constraints is pivotal for the development of fair and trustworthy machine learning models. In this paper, we introduce \sys, a framework tha…
A neural network kernel decomposition for learning multiple steady states in parameterized dynamical systems
Yimeng Zhang, Alexander Cloninger, Bo Li +1
We develop a data-driven machine learning approach to identifying parameters with steady-state solutions, locating such solutions, and determining their linear stability for system…
On a Generalization of Wasserstein Distance and the Beckmann Problem to Connection Graphs
Sawyer Robertson, Dhruv Kohli, Gal Mishne +1
We propose a model of optimal parallel transport between vector fields on a connection graph, which consists of a weighted graph along with a map from its edges to an orthogonal gr…