133 citations · 422 across the 31 of their papers we have counts for
34 papers
Beyond Decodability: Reconstructing Language Model Representations with an Encoding Probe
Gaofei Shen, Martijn Bentum, Tomas O. Lentz +2
Probing is widely used to study which features can be decoded from language model representations. However, the common decoding probe approach has two limitations that we aim to so…
On the reliability of feature attribution methods for speech classification
Gaofei Shen, Hosein Mohebbi, Arianna Bisazza +2
As the capabilities of large-scale pre-trained models evolve, understanding the determinants of their outputs becomes more important. Feature attribution aims to reveal which parts…
Co-creation for Sign Language Processing and Machine Translation
Lisa Lepp, Dimitar Shterionov, Mirella De Sisto +1
Sign language machine translation (SLMT) -- the task of automatically translating between sign and spoken languages or between sign languages -- is a complex task within the field…
QE4PE: Word-level Quality Estimation for Human Post-Editing
Gabriele Sarti, Vilém Zouhar, Grzegorz Chrupała +3
Word-level quality estimation (QE) methods aim to detect erroneous spans in machine translations, which can direct and facilitate human post-editing. While the accuracy of word-lev…
Post-hoc Disentanglement of Textual and Acoustic Features in Pre-trained Speech Encoders
Hosein Mohebbi, Grzegorz Chrupała, Willem Zuidema +2
Pre-trained speech models build entangled internal representations, which capture a variety of features (e.g., pitch, loudness, syntax, or semantics of an utterance) in a distribut…
Encoding of lexical tone in self-supervised models of spoken language
Gaofei Shen, Michaela Watkins, Afra Alishahi +2
Interpretability research has shown that self-supervised Spoken Language Models (SLMs) encode a wide variety of features in human speech from the acoustic, phonetic, phonological,…
Homophone Disambiguation Reveals Patterns of Context Mixing in Speech Transformers
Hosein Mohebbi, Grzegorz Chrupała, Willem Zuidema +1
Transformers have become a key architecture in speech processing, but our understanding of how they build up representations of acoustic and linguistic structure is limited. In thi…
Quantifying the Plausibility of Context Reliance in Neural Machine Translation
Gabriele Sarti, Grzegorz Chrupała, Malvina Nissim +1
Establishing whether language models can use contextual information in a human-plausible way is important to ensure their trustworthiness in real-world settings. However, the quest…
Wave to Syntax: Probing spoken language models for syntax
Gaofei Shen, Afra Alishahi, Arianna Bisazza +1
Understanding which information is encoded in deep models of spoken and written language has been the focus of much research in recent years, as it is crucial for debugging and imp…
Putting Natural in Natural Language Processing
Grzegorz Chrupała
Human language is firstly spoken and only secondarily written. Text, however, is a very convenient and efficient representation of language, and modern civilization has made it ubi…
Quantifying Context Mixing in Transformers
Hosein Mohebbi, Willem Zuidema, Grzegorz Chrupała +1
Self-attention weights and their transformed variants have been the main source of information for analyzing token-to-token interactions in Transformer-based models. But despite th…
Learning English with Peppa Pig
Mitja Nikolaus, Afra Alishahi, Grzegorz Chrupała
Recent computational models of the acquisition of spoken language via grounding in perception exploit associations between the spoken and visual modalities and learn to represent s…
Cyberbullying Classifiers are Sensitive to Model-Agnostic Perturbations
Chris Emmery, Ákos Kádár, Grzegorz Chrupała +1
A limited amount of studies investigates the role of model-agnostic adversarial behavior in toxic content classification. As toxicity classifiers predominantly rely on lexical cues…
ZR-2021VG: Zero-Resource Speech Challenge, Visually-Grounded Language Modelling track, 2021 edition
Afra Alishahi, Grzegorz Chrupała, Alejandrina Cristia +5
We present the visually-grounded language modelling track that was introduced in the Zero-Resource Speech challenge, 2021 edition, 2nd round. We motivate the new track and discuss…
Discrete representations in neural models of spoken language
Bertrand Higy, Lieke Gelderloos, Afra Alishahi +1
The distributed and continuous representations used by neural networks are at odds with representations employed in linguistics, which are typically symbolic. Vector quantization h…
Visually grounded models of spoken language: A survey of datasets, architectures and evaluation techniques
Grzegorz Chrupała
This survey provides an overview of the evolution of visually grounded models of spoken language over the last 20 years. Such models are inspired by the observation that when child…
Adversarial Stylometry in the Wild: Transferable Lexical Substitution Attacks on Author Profiling
Chris Emmery, Ákos Kádár, Grzegorz Chrupała
Written language contains stylistic cues that can be exploited to automatically infer a variety of potentially sensitive author information. Adversarial stylometry intends to attac…
Textual Supervision for Visually Grounded Spoken Language Understanding
Bertrand Higy, Desmond Elliott, Grzegorz Chrupała
Visually-grounded models of spoken language understanding extract semantic information directly from speech, without relying on transcriptions. This is useful for low-resource lang…
Analyzing analytical methods: The case of phonology in neural models of spoken language
Grzegorz Chrupała, Bertrand Higy, Afra Alishahi
Given the fast development of analysis techniques for NLP and speech processing systems, few systematic studies have been conducted to compare the strengths and weaknesses of each…
Learning to Understand Child-directed and Adult-directed Speech
Lieke Gelderloos, Grzegorz Chrupała, Afra Alishahi
Speech directed to children differs from adult-directed speech in linguistic aspects such as repetition, word choice, and sentence length, as well as in aspects of the speech signa…
Bootstrapping Disjoint Datasets for Multilingual Multimodal Representation Learning
Ákos Kádár, Grzegorz Chrupała, Afra Alishahi +1
Recent work has highlighted the advantage of jointly learning grounded sentence representations from multiple languages. However, the data used in these studies has been limited to…
Correlating neural and symbolic representations of language
Grzegorz Chrupała, Afra Alishahi
Analysis methods which enable us to better understand the representations and functioning of neural models of language are increasingly needed as deep learning becomes the dominant…
Analyzing and Interpreting Neural Networks for NLP: A Report on the First BlackboxNLP Workshop
Afra Alishahi, Grzegorz Chrupała, Tal Linzen
The EMNLP 2018 workshop BlackboxNLP was dedicated to resources and techniques specifically developed for analyzing and understanding the inner-workings and representations acquired…
Symbolic inductive bias for visually grounded learning of spoken language
Grzegorz Chrupała
A widespread approach to processing spoken language is to first automatically transcribe it into text. An alternative is to use an end-to-end approach: recent works have proposed t…