4 papers
Small-Scale Experiments: Are We There Yet?
Nicholas Lourie, Kyunghyun Cho, Karen Ullrich +1
Scaling laws promised cost-effective experiments; six years later, they have yet to fully deliver. Instead, researchers have found them unreliable at small scales (starting at 4M p…
Context-weighted Discrete Flow Matching
Daniil Cherniavskii, Daniel Severo, Karen Ullrich
Discrete flow matching provides a flexible framework for generative modeling on discrete structures. However, the standard factorized training objective exposes the model to target…
On the Challenges and Opportunities in Generative AI
Laura Manduchi, Clara Meister, Kushagra Pandey +23
The field of deep generative modeling has grown rapidly in the last few years. With the availability of massive amounts of training data coupled with advances in scalable unsupervi…
Lossless Compression of Vector IDs for Approximate Nearest Neighbor Search
Daniel Severo, Giuseppe Ottaviano, Matthew Muckley +2
Approximate nearest neighbor search for vectors relies on indexes that are most often accessed from RAM. Therefore, storage is the factor limiting the size of the database that can…