147 citations · 719 across the 43 of their papers we have counts for
5 papers · 1 filter
Divergence controls entropy in distillation
Nicolas Zucchet, Scott W. Linderman
Distillation has become a core primitive of large language model training, but its properties are not yet well understood. We take an entropic perspective, studying how the entropy…
Switching Linear Attention
Hyun Dong Lee, Xavier Gonzalez, Nicolas Zucchet +3
Designing expressive sequence layers with efficient inference remains a central challenge in modern machine learning. Standard softmax attention achieves excellent sequence modelin…
Language models suffer from a curse of ambiguity
Nicolas Zucchet, Hyun Dong Lee, Scott Linderman
Large language models increasingly rely on sampling as a driver of their own improvement, making the fidelity of their learned distributions more critical than ever. Yet, not all d…
Learning Biophysical Models of Large-Scale Multineuronal Data to Enable Precise Neurostimulation
Amrith Lotlikar, Ian Christopher Tanoh, Praful Vasireddy +9
Multi-compartment Hodgkin-Huxley (HH) models provide a principled framework for predicting neural dynamics and responses to electrical stimulation. However, fitting HH biophysical…
Closing the Approximation Gap in Simulation-free Latent SDEs
Henry D. Smith, Brian L. Trippe, Scott W. Linderman
Recovering dynamical systems from noisy observations is a recurring challenge across scientific domains, including neuroscience and physics. Latent stochastic differential equation…