310 citations · 436 across the 15 of their papers we have counts for
5 papers · 1 filter
Universal Spectral Tokenization via Self-Supervised Panchromatic Representation Learning
Jeff Shen, Francois Lanusse, Liam Holden Parker +24
Sequential scientific data span many resolutions and domains, and unifying them into a common representation is a key step toward developing foundation models for the sciences. Ast…
AION-1: Omnimodal Foundation Model for Astronomical Sciences
Liam Parker, Francois Lanusse, Jeff Shen +24
While foundation models have shown promise across a variety of fields, astronomy still lacks a unified framework for joint modeling across its highly diverse data modalities. In th…
Hyperparameter Loss Surfaces Are Simple Near their Optima
Nicholas Lourie, He He, Kyunghyun Cho
Hyperparameters greatly impact models' capabilities; however, modern models are too large for extensive search. Instead, researchers design recipes that train well across scales ba…
Scaling Laws Are Unreliable for Downstream Tasks: A Reality Check
Nicholas Lourie, Michael Y. Hu, Kyunghyun Cho
Downstream scaling laws aim to predict task performance at larger scales from the model's performance at smaller scales. Whether such prediction should be possible is unclear: some…
What's In Your Field? Mapping Scientific Research with Knowledge Graphs and Large Language Models
Abhipsha Das, Nicholas Lourie, Siavash Golkar +1
The scientific literature's exponential growth makes it increasingly challenging to navigate and synthesize knowledge across disciplines. Large language models (LLMs) are powerful…