4 papers
Semantic Smoothing for Language Models via Distribution Estimation and Embeddings
Haricharan Balasundaram, Swathi Shree Narashiman, Pranay Mathur +1
We propose semantic smoothing, a smoothing method for language models that uses embeddings to share statistical observations across semantically similar contexts. The starting poin…
Distribution Estimation with Side Information
Haricharan Balasundaram, Andrew Thangaraj
We consider the classical problem of discrete distribution estimation using i.i.d. samples in a novel scenario where additional side information is available on the distribution. I…
Rate of Model Collapse in Recursive Training
Ananda Theertha Suresh, Andrew Thangaraj, Aditya Nanda Kishore Khandavally
Given the ease of creating synthetic data from machine learning models, new models can be potentially trained on synthetic data generated by previous models. This recursive trainin…
Just Wing It: Near-Optimal Estimation of Missing Mass in a Markovian Sequence
Ashwin Pananjady, Vidya Muthukumar, Andrew Thangaraj
We study the problem of estimating the stationary mass -- also called the unigram mass -- that is missing from a single trajectory of a discrete-time, ergodic Markov chain. This pr…