39 citations · 48 across the 11 of their papers we have counts for
4 papers · 1 filter
The Efficiency Gap in Byte Modeling
Celine Lee, Jing Nathan Yan, Chen Liang +9
Modern language models have historically relied on two dominant design choices: subword tokenization and autoregressive (AR) ordering. These design decisions bake in priors that di…
Preparing Lessons for Progressive Training on Language Models
Yu Pan, Ye Yuan, Yichun Yin +6
The rapid progress of Transformers in artificial intelligence has come at the cost of increased resource consumption and greenhouse gas emissions due to growing model sizes. Prior…
A Finite-Particle Convergence Rate for Stein Variational Gradient Descent
Jiaxin Shi, Lester Mackey
We provide the first finite-particle convergence rate for Stein variational gradient descent (SVGD), a popular algorithm for approximating a probability distribution with a collect…
Neural Eigenfunctions Are Structured Representation Learners
Zhijie Deng, Jiaxin Shi, Hao Zhang +3
This paper introduces a structured, adaptive-length deep representation called Neural Eigenmap. Unlike prior spectral methods such as Laplacian Eigenmap that operate in a nonparame…