From the 1 of 4 linked papers with an AI index.
4 papers
Domain-Aware Scaling Laws Uncover Data Synergy
Kimia Hamidieh, Lester Mackey, David Alvarez-Melis
The paper defines and measures how mixing data from different domains during language model pretraining can produce synergistic or interfering effects, and shows that accounting fo…
Express Language Modeling
Albert Gong, Annabelle Michael Carrell, Raaz Dwivedi +1
We introduce a new tool, Express, for converting a non-causal attention approximation into a causal approximation with matching approximation guarantees. When combined with the sta…
Integrating chemical structures as treatments improves representations of microscopy images for morphological profiling
Yemin Yu, Emre Hayir, Neil Tenenholtz +5
Recent advances in self-supervised deep learning have improved our ability to quantify cellular morphological changes in high-throughput microscopy screens, a process known as morp…
KerJEPA: Kernel Discrepancies for Euclidean Self-Supervised Learning
Eric Zimmermann, Harley Wiltzer, Justin Szeto +2
Recent breakthroughs in self-supervised Joint-Embedding Predictive Architectures (JEPAs) have established that regularizing Euclidean representations toward isotropic Gaussian prio…