8 citations · 8 across the 2 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Ori: Drift-Aware Streaming -PCA for Very Low-Rank LLM Training
Xinyu Luo, Jincheng Zhou, Mengbo Wang +5
A prominent family of low-rank LLM optimizers, pioneered by GaLore, performs gradient-subspace projection: at each step, they project the gradient into a low-dimensional subspace a…
cs.LG2023★ 8 cited
Understanding the Natural Language of DNA using Encoder-Decoder Foundation Models with Byte-level Precision
Aditya Malusare, Harish Kothandaraman, Dipesh Tamboli +2
This paper presents the Ensemble Nucleotide Byte-level Encoder-Decoder (ENBED) foundation model, analyzing DNA sequences at byte-level precision with an encoder-decoder Transformer…