10 citations · 30 across the 19 of their papers we have counts for
4 papers · 1 filter
LZ Penalty: An information-theoretic repetition penalty for autoregressive language models
Antonio A. Ginart, Naveen Kodali, Jason Lee +3
We introduce the LZ penalty, a penalty specialized for reducing degenerate repetitions in autoregressive language models without loss of capability. The penalty is based on the cod…
PAC Reinforcement Learning for Predictive State Representations
Wenhao Zhan, Masatoshi Uehara, Wen Sun +1
In this paper we study online Reinforcement Learning (RL) in partially observable dynamical systems. We focus on the Predictive State Representations (PSRs) model, which is an expr…
Neural Networks can Learn Representations with Gradient Descent
Alex Damian, Jason D. Lee, Mahdi Soltanolkotabi
Significant theoretical work has established that in specific regimes, neural networks trained by gradient descent behave like kernel methods. However, in practice, it is known tha…
Sketching Meets Random Projection in the Dual: A Provable Recovery Algorithm for Big and High-dimensional Data
Jialei Wang, Jason D. Lee, Mehrdad Mahdavi +2
Sketching techniques have become popular for scaling up machine learning algorithms by reducing the sample size or dimensionality of massive data sets, while still maintaining the…