83 citations · 102 across the 11 of their papers we have counts for
13 papers
The Features at Convergence Theorem: a first-principles alternative to the Neural Feature Ansatz for how networks learn representations
Enric Boix-Adsera, Neil Mallinar, James B. Simon +1
It is a central challenge in deep learning to understand how neural networks learn representations. A leading approach is the Neural Feature Ansatz (NFA) (Radhakrishnan et al. 2024…
A Scalable Framework for Evaluating Health Language Models
Neil Mallinar, A. Ali Heydari, Xin Liu +10
Large language models (LLMs) have emerged as powerful tools for analyzing complex datasets. Recent studies demonstrate their potential to generate useful, personalized responses wh…
Eigenvectors of the De Bruijn Graph Laplacian: A Natural Basis for the Cut and Cycle Space
Anthony Philippakis, Neil Mallinar, Parthe Pandit +1
We study the Laplacian of the undirected De Bruijn graph over an alphabet of order . While the eigenvalues of this Laplacian were found in 1998 by Delorme and Tillich [1], a…
Emergence in non-neural models: grokking modular arithmetic via average gradient outer product
Neil Mallinar, Daniel Beaglehole, Libin Zhu +3
Neural networks trained to solve modular arithmetic tasks exhibit grokking, a phenomenon where the test accuracy starts improving long after the model achieves 100% training accura…
Minimum-Norm Interpolation Under Covariate Shift
Neil Mallinar, Austin Zane, Spencer Frei +1
Transfer learning is a critical part of real-world machine learning deployments and has been extensively studied in experimental works with overparameterized neural networks. Howev…
The Calibration Generalization Gap
A. Michael Carrell, Neil Mallinar, James Lucas +1
Calibration is a fundamental property of a good predictive model: it requires that the model predicts correctly in proportion to its confidence. Modern neural networks, however, pr…
Benign, Tempered, or Catastrophic: A Taxonomy of Overfitting
Neil Mallinar, James B. Simon, Amirhesam Abedsoltan +3
The practical success of overparameterized neural networks has motivated the recent scientific study of interpolating methods, which perfectly fit their training data. Certain inte…
Iterative Data Programming for Expanding Text Classification Corpora
Neil Mallinar, Abhishek Shah, Tin Kam Ho +2
Real-world text classification tasks often require many labeled training examples that are expensive to obtain. Recent advancements in machine teaching, specifically the data progr…
Multi-Frame Cross-Entropy Training for Convolutional Neural Networks in Speech Recognition
Tom Sercu, Neil Mallinar
We introduce Multi-Frame Cross-Entropy training (MFCE) for convolutional neural network acoustic models. Recognizing that similar to RNNs, CNNs are in nature sequence models that t…
Bootstrapping Conversational Agents With Weak Supervision
Neil Mallinar, Abhishek Shah, Rajendra Ugrani +9
Many conversational agents in the market today follow a standard bot development framework which requires training intent classifiers to recognize user input. The need to create a…
Big-Little Net: An Efficient Multi-Scale Feature Representation for Visual and Speech Recognition
Chun-Fu Chen, Quanfu Fan, Neil Mallinar +2
In this paper, we propose a novel Convolutional Neural Network (CNN) architecture for learning multi-scale feature representations with good tradeoffs between speed and accuracy. T…
Deep Canonically Correlated LSTMs
Neil Mallinar, Corbin Rosset
We examine Deep Canonically Correlated LSTMs as a way to learn nonlinear transformations of variable length sequences and embed them into a correlated, fixed dimensional space. We…
Probabilistic Cross-Identification of Galaxies with Realistic Clustering
Neil Mallinar, Tamas Budavari, Gerard Lemson
Probabilistic cross-identification has been successfully applied to a number of problems in astronomy from matching simple point sources to associating stars with unknown proper mo…