On convex decision regions in deep network representations

Abstract

Current work on human-machine alignment aims at understanding machine-learned latent spaces and their correspondence to human representations. G{ä}rdenfors' conceptual spaces is a prominent framework for understanding human representations. Convexity of object regions in conceptual spaces is argued to promote generalizability, few-shot learning, and interpersonal alignment. Based on these insights, we investigate the notion of convexity of concept regions in machine-learned latent spaces. We develop a set of tools for measuring convexity in sampled data and evaluate emergent convexity in layered representations of state-of-the-art deep networks. We show that convexity is robust to basic re-parametrization and, hence, meaningful as a quality of machine-learned latent spaces. We find that approximate convexity is pervasive in neural representations in multiple application domains, including models of images, audio, human activity, text, and medical images. Generally, we observe that fine-tuning increases the convexity of label regions. We find evidence that pretraining convexity of class label regions predicts subsequent fine-tuning performance.

References in corpus (13)

• SMOTE: Synthetic Minority Over-sampling Technique
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
• DINOv2: Learning Robust Visual Features without Supervision
• Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)
• What do Neural Machine Translation Models Learn about Morphology?
• data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language
• The geometry of hidden representations of large transformer models
• Convexity in complex networks
• Feature-Based Interpolation and Geodesics in the Latent Spaces of Generative Models
• Emergence of Separable Manifolds in Deep Language Representations
• Improving neural network representations using human similarity judgments
• Convexity-based Pruning of Speech Representation Models