Learning is Forgetting: LLM Training As Lossy Compression

Henry C. Conklin, Tom Hosking, Tan Yi-Chern +5

Despite the increasing prevalence of large language models (LLMs), we still have a limited understanding of how their representational spaces are structured. This limits our abilit…