7 citations · 8 across the 6 of their papers we have counts for
6 papers
LLM Probing with Contrastive Eigenproblems: Improving Understanding and Applicability of CCS
Stefan F. Schouten, Peter Bloem
Contrast-Consistent Search (CCS) is an unsupervised probing method able to test whether large language models represent binary features, such as sentence truth, in their internal a…
Truth-value judgment in language models: 'truth directions' are context sensitive
Stefan F. Schouten, Peter Bloem, Ilia Markov +1
Recent work has demonstrated that the latent spaces of large language models (LLMs) contain directions predictive of the truth of sentences. Multiple methods recover such direction…
Reasoning about Ambiguous Definite Descriptions
Stefan F. Schouten, Peter Bloem, Ilia Markov +1
Natural language reasoning plays an increasingly important role in improving language models' ability to solve complex language understanding tasks. An interesting use case for rea…
Cross-Domain Toxic Spans Detection
Stefan F. Schouten, Baran Barbarestani, Wondimagegnhue Tufa +2
Given the dynamic nature of toxic language use, automated methods for detecting toxic spans are likely to encounter distributional shift. To explore this phenomenon, we evaluate th…
A Song of (Dis)agreement: Evaluating the Evaluation of Explainable Artificial Intelligence in Natural Language Processing
Michael Neely, Stefan F. Schouten, Maurits Bleeker +1
There has been significant debate in the NLP community about whether or not attention weights can be used as an explanation - a mechanism for interpreting how important each input…
Order in the Court: Explainable AI Methods Prone to Disagreement
Michael Neely, Stefan F. Schouten, Maurits J. R. Bleeker +1
By computing the rank correlation between attention weights and feature-additive explanation methods, previous analyses either invalidate or support the role of attention-based exp…