4 papers · 1 filter
Do Prompts Reshape Representations? An Empirical Study of Prompting Effects on Embeddings
Cesar Gonzalez-Gutierrez, Dirk Hovy
Prompting is a common approach for leveraging LMs in zero-shot settings. However, the underlying mechanisms that enable LMs to perform diverse tasks without task-specific supervisi…
Domain Pre-training Impact on Representations
Cesar Gonzalez-Gutierrez, Ariadna Quattoni
This empirical study analyzes the effects of the pre-training corpus on the quality of learned transformer representations. We focus on the representation quality induced solely th…
Analyzing Text Representations by Measuring Task Alignment
Cesar Gonzalez-Gutierrez, Audi Primadhanty, Francesco Cazzaro +1
Textual representations based on pre-trained language models are key, especially in few-shot learning scenarios. What makes a representation good for text classification? Is it due…
Analyzing Text Representations under Tight Annotation Budgets: Measuring Structural Alignment
César González-Gutiérrez, Audi Primadhanty, Francesco Cazzaro +1
Annotating large collections of textual data can be time consuming and expensive. That is why the ability to train models with limited annotation budgets is of great importance. In…