14 citations · 75 across the 28 of their papers we have counts for
3 papers · 1 filter
To Infinity and Beyond: Tool-Use Unlocks Length Generalization in State Space Models
Eran Malach, Omid Saremi, Sinead Williamson +5
State Space Models (SSMs) have become the leading alternative to Transformers for sequence modeling. Their primary advantage is efficiency in long-context and long-form generation,…
Rethinking JEPA: Compute-Efficient Video SSL with Frozen Teachers
Xianhang Li, Chen Huang, Chun-Liang Li +4
Video Joint Embedding Predictive Architectures (V-JEPA) learn generalizable off-the-shelf video representation by predicting masked regions in latent space with an exponential movi…
Distillation Scaling Laws
Dan Busbridge, Amitis Shidani, Floris Weers +3
We propose a distillation scaling law that estimates distilled model performance based on a compute budget and its allocation between the student and teacher. Our findings mitigate…