1.1k citations · 3.2k across the 12 of their papers we have counts for
19 papers
Consensus, dissensus and synergy between clinicians and specialist foundation models in radiology report generation
Ryutaro Tanno, David G. T. Barrett, Andrew Sellergren +23
Radiology reports are an instrumental part of modern medicine, informing key clinical decisions such as diagnosis and treatment. The worldwide shortage of radiologists, however, re…
Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models
Maribeth Rauh, John Mellor, Jonathan Uesato +9
Large language models produce human-like text that drive a growing number of applications. However, recent literature and, increasingly, real world observations, have demonstrated…
Training Compute-Optimal Large Language Models
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch +19
We investigate the optimal model size and number of tokens for training a transformer language model under a given compute budget. We find that current large language models are si…
Competition-Level Code Generation with AlphaCode
Yujia Li, David Choi, Junyoung Chung +23
Programming is a powerful and ubiquitous problem-solving tool. Developing systems that can assist programmers or even generate programs independently could make programming more pr…
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Jack W. Rae, Sebastian Borgeaud, Trevor Cai +77
Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.…
Challenges in Detoxifying Language Models
Johannes Welbl, Amelia Glaese, Jonathan Uesato +7
Large language models (LM) generate remarkably fluent text and can be efficiently adapted across NLP tasks. Measuring and guaranteeing the quality of generated text in terms of saf…
Evaluating the Apperception Engine
Richard Evans, Jose Hernandez-Orallo, Johannes Welbl +2
The Apperception Engine is an unsupervised learning system. Given a sequence of sensory inputs, it constructs a symbolic causal theory that both explains the sensory sequence and a…
Undersensitivity in Neural Reading Comprehension
Johannes Welbl, Pasquale Minervini, Max Bartolo +2
Current reading comprehension models generalise well to in-distribution test sets, yet perform poorly on adversarially selected inputs. Most prior work on adversarial inputs studie…
Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension
Max Bartolo, Alastair Roberts, Johannes Welbl +2
Innovations in annotation methodology have been a catalyst for Reading Comprehension (RC) datasets and models. One recent trend to challenge current RC models is to involve a model…
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation
Po-Sen Huang, Huan Zhang, Ray Jiang +6
Advances in language modeling architectures and the availability of large text corpora have driven progress in automatic text generation. While this results in models capable of ge…
Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagation
Po-Sen Huang, Robert Stanforth, Johannes Welbl +5
Neural networks are part of many contemporary NLP systems, yet their empirical successes come at the price of vulnerability to adversarial attacks. Previous work has used adversari…
Jack the Reader - A Machine Reading Framework
Dirk Weissenborn, Pasquale Minervini, Tim Dettmers +8
Many Machine Reading and Natural Language Understanding tasks require reading supporting text in order to answer questions. For example, in Question Answering, the supporting text…
Constructing Datasets for Multi-hop Reading Comprehension Across Documents
Johannes Welbl, Pontus Stenetorp, Sebastian Riedel
Most Reading Comprehension methods limit themselves to queries which can be answered using a single sentence, paragraph, or document. Enabling models to combine disjoint pieces of…
Crowdsourcing Multiple Choice Science Questions
Johannes Welbl, Nelson F. Liu, Matt Gardner
We present a novel method for obtaining high-quality, domain-targeted multiple choice questions from crowd workers. Generating these questions can be difficult without trading away…
Frustratingly Short Attention Spans in Neural Language Modeling
Michał Daniluk, Tim Rocktäschel, Johannes Welbl +1
Neural language models predict the next token using a latent representation of the immediate token history. Recently, various methods for augmenting neural language models with an…
Knowledge Graph Completion via Complex Tensor Factorization
Théo Trouillon, Christopher R. Dance, Johannes Welbl +3
In statistical relational learning, knowledge graph completion deals with automatically understanding the structure of large knowledge graphs---labeled directed graphs---and predic…
Complex Embeddings for Simple Link Prediction
Théo Trouillon, Johannes Welbl, Sebastian Riedel +2
In statistical relational learning, the link prediction problem is key to automatically understand the structure of large knowledge bases. As in previous studies, we propose to sol…
A Factorization Machine Framework for Testing Bigram Embeddings in Knowledgebase Completion
Johannes Welbl, Guillaume Bouchard, Sebastian Riedel
Embedding-based Knowledge Base Completion models have so far mostly combined distributed representations of individual entities or relations to compute truth scores of missing link…
Neural Random Forests
Gérard Biau, Erwan Scornet, Johannes Welbl
Given an ensemble of randomized regression trees, it is possible to restructure them as a collection of multilayered neural networks with particular connection weights. Following t…