4 citations · 13 across the 32 of their papers we have counts for
34 papers
Improving Generative Model Self-Training with Geometrically Modified Outputs
Patrick Batsell, Thomas Walker, Richard Baraniuk
Self-training generative models - the continued improvement of a model using its own outputs - is becoming increasingly important as high-quality training data becomes scarce. Howe…
Behavioral Monitoring of JEPA World Models with Jacobian Centroids
Thomas Walker, Randall Balestriero, Richard Baraniuk
Detecting failures in World Model (WM)-based planning requires monitoring whether the model is behaviorally aligned with the current task, which in turn requires studying its inter…
Finsler Geometry, Graph Neural Networks, and You
T. Mitchell Roddenberry, Richard G. Baraniuk
Graph neural network architectures based on the graph Laplacian approximate the Laplace-Beltrami operator, thus limiting their application to isotropic operators. As a nonlinear al…
The Geometric Structure of Models Learning Sparse Data
Thomas Walker, T. Mitchell Roddenberry, Ahmed Imtiaz Humayun +2
The manifold hypothesis (MH) is often used to explain how machine learning can overcome the curse of dimensionality. However, the MH is only applicable in regimes where the trainin…
The Linear Centroids Hypothesis: Features as Directions Learned by Local Experts
Thomas Walker, Ahmed Imtiaz Humayun, Randall Balestriero +1
The Linear Representation Hypothesis (LRH) identifies features of a trained deep network (DN) as linear directions in the activation spaces, i.e., output spaces of intermediate lay…
Minimizing Collateral Damage in Activation Steering
Tam Nguyen, Tu Anh Nguyen, Sina Alemohammad +1
Activation steering is a method for controlling Large Language Model (LLM) behavior by intervening in its internal representations to increase the alignment with a specific target…
Leakage and Second-Order Dynamics Improve Hippocampal RNN Replay
Josue Casco-Rodriguez, Nanda H. Krishna, Richard G. Baraniuk
Biological neural networks (like the hippocampus) can internally generate "replay" resembling stimulus-driven activity. Recent computational models of replay use noisy recurrent ne…
Rates and architectures for learning geometrically non-trivial operators
T. Mitchell Roddenberry, Leo Tzou, Ivan Dokmanić +2
Deep learning methods have proven capable of recovering operators between high-dimensional spaces, such as solution maps of PDEs and similar objects in mathematical physics, from v…
Neon: Negative Extrapolation From Self-Training Improves Image Generation
Sina Alemohammad, Zhangyang Wang, Richard G. Baraniuk
Scaling generative AI models is bottlenecked by the scarcity of high-quality training data. The ease of synthesizing from a generative model suggests using (unverified) synthetic d…
GrokAlign: Geometric Characterisation and Acceleration of Grokking
Thomas Walker, Ahmed Imtiaz Humayun, Randall Balestriero +1
A key challenge for the machine learning community is to understand and accelerate the training dynamics of deep networks that lead to delayed generalisation and emergent robustnes…
W4S4: WaLRUS Meets S4 for Long-Range Sequence Modeling
Hossein Babaei, Mel White, Richard G. Baraniuk
State Space Models (SSMs) have emerged as powerful components for sequence modeling, enabling efficient handling of long-range dependencies via linear recurrence and convolutional…
WaLRUS: Wavelets for Long-range Representation Using SSMs
Hossein Babaei, Mel White, Sina Alemohammad +1
State-Space Models (SSMs) have proven to be powerful tools for modeling long-range dependencies in sequential data. While the recent method known as HiPPO has demonstrated strong p…
SaFARi: State-Space Models for Frame-Agnostic Representation
Hossein Babaei, Mel White, Sina Alemohammad +1
State-Space Models (SSMs) have re-emerged as a powerful tool for online function approximation, and as the backbone of machine learning models for long-range dependent data. Howeve…
Improving Routing in Sparse Mixture of Experts with Graph of Tokens
Tam Nguyen, Ngoc N. Tran, Khai Nguyen +1
Sparse Mixture of Experts (SMoE) has emerged as a key to achieving unprecedented scalability in deep learning. By activating only a small subset of parameters per sample, SMoE achi…
Estimating the Number and Locations of Boundaries in Reverberant Environments with Deep Learning
Toros Arikan, Luca M. Chackalackal, Fatima Ahsan +4
Underwater acoustic environment estimation is a challenging but important task for remote sensing scenarios. Current estimation methods require high signal strength and a solution…
MazeNet: An Accurate, Fast, and Scalable Deep Learning Solution for Steiner Minimum Trees
Gabriel Díaz Ramos, Toros Arikan, Richard G. Baraniuk
The Obstacle Avoiding Rectilinear Steiner Minimum Tree (OARSMT) problem, which seeks the shortest interconnection of a given number of terminals in a rectilinear plane while avoidi…
Learning Transferable Features for Implicit Neural Representations
Kushal Vyas, Ahmed Imtiaz Humayun, Aniket Dashpute +3
Implicit neural representations (INRs) have demonstrated success in a variety of applications, including inverse problems and neural rendering. An INR is typically trained to captu…
Self-Improving Diffusion Models with Synthetic Data
Sina Alemohammad, Ahmed Imtiaz Humayun, Shruti Agarwal +2
The artificial intelligence (AI) world is running out of real data for training increasingly large generative models, resulting in accelerating pressure to train on synthetic data.…
On the Geometry of Deep Learning
Randall Balestriero, Ahmed Imtiaz Humayun, Richard Baraniuk
In this paper, we overview one promising avenue of progress at the mathematical foundation of deep learning: the connection between deep networks and function approximation by affi…
Synthetic Context Generation for Question Generation
Naiming Liu, Zichao Wang, Richard Baraniuk
Despite rapid advancements in large language models (LLMs), QG remains a challenging problem due to its complicated process, open-ended nature, and the diverse settings in which qu…
Mitigating over-exploration in latent space optimization using LES
Omer Ronen, Ahmed Imtiaz Humayun, Richard Baraniuk +2
We develop Latent Exploration Score (LES) to mitigate over-exploration in Latent Space Optimization (LSO), a popular method for solving black-box discrete optimization problems. LS…
Improving Fairness and Mitigating MADness in Generative Models
Paul Mayer, Lorenzo Luzi, Ali Siahkoohi +2
Generative models unfairly penalize data belonging to minority classes, suffer from model autophagy disorder (MADness), and learn biased estimates of the underlying distribution pa…
Automated Long Answer Grading with RiceChem Dataset
Shashank Sonkar, Kangqi Ni, Lesa Tran Lu +3
We introduce a new area of study in the field of educational Natural Language Processing: Automated Long Answer Grading (ALAG). Distinguishing itself from Automated Short Answer Gr…
Marking: Visual Grading with Highlighting Errors and Annotating Missing Bits
Shashank Sonkar, Naiming Liu, Debshila B. Mallick +1
In this paper, we introduce "Marking", a novel grading task that enhances automated grading systems by performing an in-depth analysis of student responses and providing students w…