21 citations · 63 across the 37 of their papers we have counts for
11 papers · 1 filter
Efficient Algorithms for Recognizing Weighted Tree-Adjoining Languages
Alexandra Butoi, Tim Vieira, Ryan Cotterell +1
The class of tree-adjoining languages can be characterized by various two-level formalisms, consisting of a context-free grammar (CFG) or pushdown automaton (PDA) controlling anoth…
Tokenization and the Noiseless Channel
Vilém Zouhar, Clara Meister, Juan Luis Gastaldi +3
Subword tokenization is a key part of many NLP pipelines. However, little is known about why some tokenizer and hyperparameter combinations lead to better downstream model performa…
Convergence and Diversity in the Control Hierarchy
Alexandra Butoi, Ryan Cotterell, David Chiang
Weir has defined a hierarchy of language classes whose second member () is generated by tree-adjoining grammars (TAG), linear indexed grammars (LIG), combinatory cat…
A Formal Perspective on Byte-Pair Encoding
Vilém Zouhar, Clara Meister, Juan Luis Gastaldi +4
Byte-Pair Encoding (BPE) is a popular algorithm used for tokenizing data in NLP, despite being devised initially as a compression method. BPE appears to be a greedy algorithm at fa…
Structured Voronoi Sampling
Afra Amini, Li Du, Ryan Cotterell
Gradient-based sampling algorithms have demonstrated their effectiveness in text generation, especially in the context of controlled text generation. However, there exists a lack o…
All Roads Lead to Rome? Exploring the Invariance of Transformers' Representations
Yuxin Ren, Qipeng Guo, Zhijing Jin +4
Transformer models bring propelling advances in various NLP tasks, thus inducing lots of interpretability research on the learned representations of the models. However, we raise a…
RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text
Wangchunshu Zhou, Yuchen Eleanor Jiang, Peng Cui +5
The fixed-size context of Transformer makes GPT models incapable of generating arbitrarily long text. In this paper, we introduce RecurrentGPT, a language-based simulacrum of the r…
Efficient Prompting via Dynamic In-Context Learning
Wangchunshu Zhou, Yuchen Eleanor Jiang, Ryan Cotterell +1
The primary way of building AI applications is shifting from training specialist models to prompting generalist models. A common practice for prompting generalist models, often ref…
Discourse Centric Evaluation of Machine Translation with a Densely Annotated Parallel Corpus
Yuchen Eleanor Jiang, Tianyu Liu, Shuming Ma +3
Several recent papers claim human parity at sentence-level Machine Translation (MT), especially in high-resource languages. Thus, in response, the MT community has, in part, shifte…
Linear-Time Modeling of Linguistic Structure: An Order-Theoretic Perspective
Tianyu Liu, Afra Amini, Mrinmaya Sachan +1
Tasks that model the relation between pairs of tokens in a string are a vital part of understanding natural language. Such tasks, in general, require exhaustive pair-wise compariso…
Controlled Text Generation with Natural Language Instructions
Wangchunshu Zhou, Yuchen Eleanor Jiang, Ethan Wilcox +2
Large language models generate fluent texts and can follow natural language instructions to solve a wide range of tasks without task-specific training. Nevertheless, it is notoriou…