1 citations · 2 across the 13 of their papers we have counts for
3 papers · 1 filter
Homophone Disambiguation Reveals Patterns of Context Mixing in Speech Transformers
Hosein Mohebbi, Grzegorz Chrupała, Willem Zuidema +1
Transformers have become a key architecture in speech processing, but our understanding of how they build up representations of acoustic and linguistic structure is limited. In thi…
DecoderLens: Layerwise Interpretation of Encoder-Decoder Transformers
Anna Langedijk, Hosein Mohebbi, Gabriele Sarti +2
In recent years, many interpretability methods have been proposed to help interpret the internal states of Transformer-models, at different levels of precision and complexity. Here…
Quantifying Context Mixing in Transformers
Hosein Mohebbi, Willem Zuidema, Grzegorz Chrupała +1
Self-attention weights and their transformed variants have been the main source of information for analyzing token-to-token interactions in Transformer-based models. But despite th…