322 citations · 544 across the 4 of their papers we have counts for
9 papers
SMYRF: Efficient Attention using Asymmetric Clustering
Giannis Daras, Nikita Kitaev, Augustus Odena +1
We propose a novel type of balanced clustering algorithm to approximate attention. Attention complexity is reduced from to , where is the sequence length.…
Unsupervised Parsing via Constituency Tests
Steven Cao, Nikita Kitaev, Dan Klein
We propose a method for unsupervised parsing based on the linguistic notion of a constituency test. One type of constituency test involves modifying the sentence via some transform…
Reformer: The Efficient Transformer
Nikita Kitaev, Łukasz Kaiser, Anselm Levskaya
Large Transformer models routinely achieve state-of-the-art results on a number of tasks but training these models can be prohibitively costly, especially on long sequences. We int…
Multilingual Alignment of Contextual Word Representations
Steven Cao, Nikita Kitaev, Dan Klein
We propose procedures for evaluating and strengthening contextual embedding alignment and show that they are useful in analyzing and improving multilingual BERT. In particular, aft…
Cross-Domain Generalization of Neural Constituency Parsers
Daniel Fried, Nikita Kitaev, Dan Klein
Neural parsers obtain state-of-the-art results on benchmark treebanks for constituency parsing -- but to what degree do they generalize to other domains? We present three results a…
KERMIT: Generative Insertion-Based Modeling for Sequences
William Chan, Nikita Kitaev, Kelvin Guu +2
We present KERMIT, a simple insertion-based approach to generative modeling for sequences and sequence pairs. KERMIT models the joint distribution and its decompositions (i.e., mar…