5 papers
Optimization Dynamics Imprint Semantic Specificity in Contrastive Embedding Norms
Ziwei Su, Junyu Ren, Victor Veitch
Contrastive embedding models trained with scale-invariant losses are typically paired with distance metrics like cosine similarity, effectively ignoring embedding magnitudes. Howev…
The Information Geometry of Softmax: Probing and Steering
Kiho Park, Todd Nief, Yo Joong Choe +1
This paper concerns the question of how AI systems encode semantic structure into the geometric structure of their representation spaces. The motivating observation is that the nat…
Incorporating Hierarchical Semantics in Sparse Autoencoder Architectures
Mark Muchane, Sean Richardson, Kiho Park +1
Sparse dictionary learning (and, in particular, sparse autoencoders) attempts to learn a set of human-understandable concepts that can explain variation on an abstract space. A bas…
Does Editing Provide Evidence for Localization?
Zihao Wang, Victor Veitch
A basic aspiration for interpretability research in large language models is to "localize" semantically meaningful behaviors to particular components within the LLM. There are vari…
The Geometry of Categorical and Hierarchical Concepts in Large Language Models
Kiho Park, Yo Joong Choe, Yibo Jiang +1
The linear representation hypothesis is the informal idea that semantic concepts are encoded as linear directions in the representation spaces of large language models (LLMs). Prev…