1.1k citations · 1.2k across the 10 of their papers we have counts for
13 papers
Knowledgeless Language Models: Suppressing Parametric Recall for Evidence-Grounded Language Modeling
Roi Cohen, Yvan Carré, Nick Lechtenbörger +5
Language models encode substantial factual knowledge in their parameters, which can lead to unreliable behavior when this knowledge is outdated, incomplete, or misaligned with the…
Subword Segmental Machine Translation: Unifying Segmentation and Target Sentence Generation
Francois Meyer, Jan Buys
Subword segmenters like BPE operate as a preprocessing step in neural machine translation and other (conditional) language models. They are applied to datasets before training, so…
University of Cape Town's WMT22 System: Multilingual Machine Translation for Southern African Languages
Khalid N. Elmadani, Francois Meyer, Jan Buys
The paper describes the University of Cape Town's submission to the constrained track of the WMT22 Shared Task: Large-Scale Machine Translation Evaluation for African Languages. Ou…
Subword Segmental Language Modelling for Nguni Languages
Francois Meyer, Jan Buys
Subwords have become the standard units of text in NLP, enabling efficient open-vocabulary models. With algorithms like byte-pair encoding (BPE), subword segmentation is viewed as…
Low-Resource Language Modelling of South African Languages
Stuart Mesham, Luc Hayward, Jared Shapiro +1
Language models are the foundation of current neural network-based models for natural language understanding and generation. However, research on the intrinsic performance of langu…
Canonical and Surface Morphological Segmentation for Nguni Languages
Tumi Moeng, Sheldon Reay, Aaron Daniels +1
Morphological Segmentation involves decomposing words into morphemes, the smallest meaning-bearing units of language. This is an important NLP task for morphologically-rich aggluti…
BottleSum: Unsupervised and Self-supervised Sentence Summarization using the Information Bottleneck Principle
Peter West, Ari Holtzman, Jan Buys +1
The principle of the Information Bottleneck (Tishby et al. 1999) is to produce a summary of information X optimized to predict some other relevant information Y. In this paper, we…
The Curious Case of Neural Text Degeneration
Ari Holtzman, Jan Buys, Li Du +2
Despite considerable advancements with deep neural language models, the enigma of neural text degeneration persists when these models are tested as text generators. The counter-int…
Learning to Write with Cooperative Discriminators
Ari Holtzman, Jan Buys, Maxwell Forbes +3
Recurrent Neural Networks (RNNs) are powerful autoregressive sequence models, but when used to generate natural language their output tends to be overly generic, repetitive, and se…
Robust Incremental Neural Semantic Graph Parsing
Jan Buys, Phil Blunsom
Parsing sentences to linguistically-expressive semantic representations is a key goal of Natural Language Processing. Yet statistical parsing has focused almost exclusively on bile…
Online Segment to Segment Neural Transduction
Lei Yu, Jan Buys, Phil Blunsom
We introduce an online neural sequence to sequence model that learns to alternate between encoding and decoding segments of the input as it is read. By independently tracking the e…
Cross-Lingual Morphological Tagging for Low-Resource Languages
Jan Buys, Jan A. Botha
Morphologically rich languages often lack the annotated linguistic resources required to develop accurate natural language processing tools. We propose models suitable for training…
A Bayesian Model for Generative Transition-based Dependency Parsing
Jan Buys, Phil Blunsom
We propose a simple, scalable, fully generative model for transition-based dependency parsing with high accuracy. The model, parameterized by Hierarchical Pitman-Yor Processes, ove…