7 citations · 24 across the 19 of their papers we have counts for
3 papers · 1 filter
BERT on a Data Diet: Finding Important Examples by Gradient-Based Pruning
Mohsen Fayyaz, Ehsan Aghazadeh, Ali Modarressi +3
Current pre-trained language models rely on large datasets for achieving state-of-the-art performance. However, past research has shown that not all examples in a dataset are equal…
GlobEnc: Quantifying Global Token Attribution by Incorporating the Whole Encoder Layer in Transformers
Ali Modarressi, Mohsen Fayyaz, Yadollah Yaghoobzadeh +1
There has been a growing interest in interpreting the underlying dynamics of Transformers. While self-attention patterns were initially deemed as the primary option, recent studies…
AdapLeR: Speeding up Inference by Adaptive Length Reduction
Ali Modarressi, Hosein Mohebbi, Mohammad Taher Pilehvar
Pre-trained language models have shown stellar performance in various downstream tasks. But, this usually comes at the cost of high latency and computation, hindering their usage i…