Publications (16)
Hyperbolic Aware Minimization: Implicit Bias for Sparsity
Tom Jacobs, Advait Gadhikar, Celia Rubio-Madrigal +1
Understanding the implicit bias of optimization algorithms is key to explaining and improving the generalization of deep models. The hyperbolic implicit bias induced by pointwise o…
Kepler Eclipsing Binary Stars. VII. The Catalog of Eclipsing Binaries Found in the Entire Kepler Data-Set
Brian Kirk, Kyle Conroy, Andrej Prša +47
The primary Kepler Mission provided nearly continuous monitoring of ~200,000 objects with unprecedented photometric precision. We present the final catalog of eclipsing binary syst…
SparseOpt: Addressing Normalization-induced Gradient Skew in Sparse Training
Mohammed Adnan, Rohan Jain, Tom Jacobs +4
Dynamic Sparse Training (DST) methods train neural networks by maintaining sparsity while dynamically adapting the network topology. Despite the promise of reduced computation, DST…
Pay Attention to Small Weights
Chao Zhou, Tom Jacobs, Advait Gadhikar +1
Finetuning large pretrained neural networks is known to be resource-intensive, both in terms of memory and computational cost. To mitigate this, a common approach is to restrict tr…
The Graphon Limit Hypothesis: Understanding Neural Network Pruning via Infinite Width Analysis
Hoang Pham, The-Anh Ta, Tom Jacobs +2
Sparse neural networks promise efficiency, yet training them effectively remains a fundamental challenge. Despite advances in pruning methods that create sparse architectures, unde…
Planetesimals Around Stars with TESS (PAST): II. An M Dwarf "Dipper" Star with a Long-Lived Disk in the TESS Continuous Viewing Zone
Eric Gaidos, Andrew W. Mann, Bárbara Rojas-Ayala +6
Studies of T Tauri disks inform planet formation theory; observations of variability due to occultation by circumstellar dust are a useful probe of unresolved, planet-forming inner…
Implicit Bias of Mirror Flow in Homogeneous Neural Networks: Sparse and Dense Feature Learning
Tom Jacobs, Guido Montufar
We study the max-margin solutions reached by mirror flow in deep neural networks with homogeneous activation functions. Extending classical results on gradient flow, we derive a no…
Giant Outer Transiting Exoplanet Mass (GOT 'EM) Survey: III. Recovery and Confirmation of a Temperate, Mildly Eccentric, Single-Transit Jupiter Orbiting TOI-2010
Christopher R. Mann, Paul A. Dalba, David Lafrenière +48
Large-scale exoplanet surveys like the TESS mission are powerful tools for discovering large numbers of exoplanet candidates. Single-transit events are commonplace within the resul…
Planet Hunters. VI: An Independent Characterization of KOI-351 and Several Long Period Planet Candidates from the Kepler Archival Data
Joseph R. Schmitt, Ji Wang, Debra A. Fischer +29
We report the discovery of 14 new transiting planet candidates in the Kepler field from the Planet Hunters citizen science program. None of these candidates overlapped with Kepler…
Never Saddle for Reparameterized Steepest Descent as Mirror Flow
Tom Jacobs, Chao Zhou, Rebekka Burkholz
How does the choice of optimization algorithm shape a model's ability to learn features? To address this question for steepest descent methods --including sign descent, which is cl…
Sign-In to the Lottery: Reparameterizing Sparse Training From Scratch
Advait Gadhikar, Tom Jacobs, Chao Zhou +1
The performance gap between training sparse neural networks from scratch (PaI) and dense-to-sparse training presents a major roadblock for efficient deep learning. According to the…
Mirror, Mirror of the Flow: How Does Regularization Shape Implicit Bias?
Tom Jacobs, Chao Zhou, Rebekka Burkholz
Implicit bias plays an important role in explaining how overparameterized models generalize well. Explicit regularization like weight decay is often employed in addition to prevent…
Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse
Shuang Liang, Tom Jacobs, Guido Montúfar
We study the implicit bias of noisy stochastic gradient descent in training wide two-layer ReLU networks for multivariate regression. In a mean-field regime, the training dynamics…
Mask in the Mirror: Implicit Sparsification
Tom Jacobs, Rebekka Burkholz
Continuous sparsification strategies are among the most effective methods for reducing the inference costs and memory demands of large-scale neural networks. A key factor in their…
Kepler's Last Planet Discoveries: Two New Planets and One Single-Transit Candidate from K2 Campaign 19
Elyse Incha, Andrew Vanderburg, Tom Jacobs +9
The Kepler space telescope was responsible for the discovery of over 2,700 confirmed exoplanets, more than half of the total number of exoplanets known today. These discoveries too…
HORST: Composing Optimizer Geometries for Sparse Transformer Training
Tom Jacobs, Rohan Jain, Rebekka Burkholz
Sparsifying transformers remains a fundamental challenge, as standard optimizers fail to simultaneously encourage sparsity and maintain training stability. Effective adaptive optim…