97 citations · 406 across the 28 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
q-bio.NC2024
Comment on Deterministic Information Bottleneck
Sarah Marzen
We make the case that although Deterministic Information Bottleneck may be a contribution to clustering, it should not be used to aid lossy compression without the addition of bloc…
cs.LG2024★ 1 cited
Transformers represent belief state geometry in their residual stream
Adam S. Shai, Sarah E. Marzen, Lucas Teixeira +2
What computational structure are we building into large language models when we train them on next-token prediction? Here, we present evidence that this structure is given by the m…