activity
20182024
most citedAnalyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned

70 citations · 95 across the 10 of their papers we have counts for

collaborators
Showing 2023Show all

2 papers · 1 filter

cs.CL2023

Neurons in Large Language Models: Dead, N-gram, Positional

Elena Voita, Javier Ferrando, Christoforos Nalmpantis

We analyze a family of large language models in such a lightweight manner that can be done on a single GPU. Specifically, we focus on the OPT family of models ranging from 125m to…

cs.CL2023

HalOmi: A Manually Annotated Benchmark for Multilingual Hallucination and Omission Detection in Machine Translation

David Dale, Elena Voita, Janice Lam +6

Hallucinations in machine translation are translations that contain information completely unrelated to the input. Omissions are translations that do not include some of the input…