322 citations · 608 across the 9 of their papers we have counts for
9 papers
SMYRF: Efficient Attention using Asymmetric Clustering
Giannis Daras, Nikita Kitaev, Augustus Odena +1
We propose a novel type of balanced clustering algorithm to approximate attention. Attention complexity is reduced from to , where is the sequence length.…
Unsupervised Parsing via Constituency Tests
Steven Cao, Nikita Kitaev, Dan Klein
We propose a method for unsupervised parsing based on the linguistic notion of a constituency test. One type of constituency test involves modifying the sentence via some transform…
Reformer: The Efficient Transformer
Nikita Kitaev, Łukasz Kaiser, Anselm Levskaya
Large Transformer models routinely achieve state-of-the-art results on a number of tasks but training these models can be prohibitively costly, especially on long sequences. We int…
Multilingual Alignment of Contextual Word Representations
Steven Cao, Nikita Kitaev, Dan Klein
We propose procedures for evaluating and strengthening contextual embedding alignment and show that they are useful in analyzing and improving multilingual BERT. In particular, aft…
Cross-Domain Generalization of Neural Constituency Parsers
Daniel Fried, Nikita Kitaev, Dan Klein
Neural parsers obtain state-of-the-art results on benchmark treebanks for constituency parsing -- but to what degree do they generalize to other domains? We present three results a…
KERMIT: Generative Insertion-Based Modeling for Sequences
William Chan, Nikita Kitaev, Kelvin Guu +2
We present KERMIT, a simple insertion-based approach to generative modeling for sequences and sequence pairs. KERMIT models the joint distribution and its decompositions (i.e., mar…
Tetra-Tagging: Word-Synchronous Parsing with Linear-Time Inference
Nikita Kitaev, Dan Klein
We present a constituency parsing algorithm that, like a supertagger, works by assigning labels to each word in a sentence. In order to maximally leverage current neural architectu…
Multilingual Constituency Parsing with Self-Attention and Pre-Training
Nikita Kitaev, Steven Cao, Dan Klein
We show that constituency parsing benefits from unsupervised pre-training across a variety of languages and a range of pre-training conditions. We first compare the benefits of no…
Constituency Parsing with a Self-Attentive Encoder
Nikita Kitaev, Dan Klein
We demonstrate that replacing an LSTM encoder with a self-attentive architecture can lead to improvements to a state-of-the-art discriminative constituency parser. The use of atten…