1k citations · 2.2k across the 22 of their papers we have counts for
12 papers · 1 filter
Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models
Maribeth Rauh, John Mellor, Jonathan Uesato +9
Large language models produce human-like text that drive a growing number of applications. However, recent literature and, increasingly, real world observations, have demonstrated…
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Jack W. Rae, Sebastian Borgeaud, Trevor Cai +77
Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.…
Ethical and social risks of harm from Language Models
Laura Weidinger, John Mellor, Maribeth Rauh +20
This paper aims to help structure the risk landscape associated with large-scale Language Models (LMs). In order to foster advances in responsible innovation, an in-depth understan…
Challenges in Detoxifying Language Models
Johannes Welbl, Amelia Glaese, Jonathan Uesato +7
Large language models (LM) generate remarkably fluent text and can be efficiently adapted across NLP tasks. Measuring and guaranteeing the quality of generated text in terms of saf…
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation
Po-Sen Huang, Huan Zhang, Ray Jiang +6
Advances in language modeling architectures and the availability of large text corpora have driven progress in automatic text generation. While this results in models capable of ge…
Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagation
Po-Sen Huang, Robert Stanforth, Johannes Welbl +5
Neural networks are part of many contemporary NLP systems, yet their empirical successes come at the price of vulnerability to adversarial attacks. Previous work has used adversari…
Neural Phrase-to-Phrase Machine Translation
Jiangtao Feng, Lingpeng Kong, Po-Sen Huang +5
In this paper, we propose Neural Phrase-to-Phrase Machine Translation (NPMT). Our model uses a phrase attention mechanism to discover relevant input (source) segments that are…
Robust Text-to-SQL Generation with Execution-Guided Decoding
Chenglong Wang, Kedar Tatwawadi, Marc Brockschmidt +4
We consider the problem of neural semantic parsing, which translates natural language questions into executable SQL queries. We introduce a new mechanism, execution guidance, to le…
Discourse-Aware Neural Rewards for Coherent Text Generation
Antoine Bosselut, Asli Celikyilmaz, Xiaodong He +3
In this paper, we investigate the use of discourse-aware rewards with reinforcement learning to guide a model to generate long, coherent text. In particular, we propose to learn ne…
Natural Language to Structured Query Generation via Meta-Learning
Po-Sen Huang, Chenglong Wang, Rishabh Singh +2
In conventional supervised training, a model is trained to fit all the training examples. However, having a monolithic model may not always be the best strategy, as examples could…
Two-Stage Synthesis Networks for Transfer Learning in Machine Comprehension
David Golub, Po-Sen Huang, Xiaodong He +1
We develop a technique for transfer learning in machine comprehension (MC) using a novel two-stage synthesis network (SynNet). Given a high-performing MC model in one domain, our t…
Towards Neural Phrase-based Machine Translation
Po-Sen Huang, Chong Wang, Sitao Huang +2
In this paper, we present Neural Phrase-based Machine Translation (NPMT). Our method explicitly models the phrase structures in output sequences using Sleep-WAke Networks (SWAN), a…