1 citations · 1 across the 18 of their papers we have counts for
8 papers · 1 filter
Transformers as Adaptive Kernel Learners across Tasks for In-Context Learning
Zhaiming Shen, Alexander Cloninger, Rongjie Lai +1
Transformers have emerged as a dominant architecture in modern machine learning, with in-context learning (ICL) enabling models to adapt to new tasks by conditioning on prompts, wi…
Residual-Stream Burden Shapes Representation Learning in Diffusion Transformers
Tongtong Liang, Siqi Kou, Ziqiao Xi +6
In diffusion-based generation, a neural network can be trained to predict the clean data, the noise, or the velocity from a noisy input. These prediction targets are interconvertib…
Transformers as Cross-Task Learners: Shared Structure Drives Sample Efficiency in In-Context Learning
Zhongjie Shi, Rongjie Lai, Alexander Cloninger +1
Transformers achieve remarkable performance by jointly learning broad families of tasks during pretraining and adapting to unseen tasks from only a short prompt. Yet a rigorous mat…
Riemannian ascent--descent for nonconvex nonconcave minimax landscapes: convergence to basin saddle points and applications to distributionally robust optimization
Rishabh Dixit, Pranav Upadrashta, Alex Cloninger
We study a class of distributionally robust optimization (DRO) problems for the statistical risk problem, formulated as minimax problems over the product of a Euclidean space and a…
Scalable Graph Coreset Selection via Greedy Sampling
Zhaiming Shen, Alexander Cloninger
Sampling representative nodes from large graphs is fundamental to graph signal processing and network analysis, yet existing methods require access to the full graph Laplacian, mak…
Unbalanced Optimal Transport Dictionary Learning for Unsupervised Hyperspectral Image Clustering
Joshua Lentz, Nicholas Karris, Alex Cloninger +1
Hyperspectral images capture vast amounts of high-dimensional spectral information about a scene, making labeling an intensive task that is resistant to out-of-the-box statistical…
Does Sparse Connectivity Improve Generalization? Convolutional Networks Below the Edge of Stability
Tongtong Liang, Esha Singh, Rahul Parhi +2
Gradient descent on overparameterized neural networks typically operates at the Edge of Stability (EoS), where the largest Hessian eigenvalue hovers around a step-size-dependent th…
Optimal Transport, Timesteppers, Newton-Krylov Methods and Steady States of Collective Particle Dynamics
Hannes Vandecasteele, Nicholas Karris, Alexander Cloninger +1
Timesteppers constitute a powerful tool in modern computational science and engineering. Although they are typically used to advance the system forward in time, they can also be vi…