243 citations · 339 across the 12 of their papers we have counts for
16 papers
Who Thinks First? Designing Productive Friction with Engage-to-Unlock GenAI
Xiaotian Su, Laura Rimell, Jiazheng Li +6
Generative AI can support writing, but frictionless access may cause cognitive offloading before users develop their own ideas. We introduce Engage-to-Unlock, a productive-friction…
Fabula: Building a Narrative Storytelling Sidekick with the Writers' Community
Piotr Mirowski, Ben Wedin, Reinald Kim Amplayo +14
We design and evaluate Fabula, an interactive app for fiction writers. Fabula uses detailed narrative plans informed by general narratological theory. Stories are structured hierar…
Scratchpad Patching: Decoupling Compute from Patch Size in Byte-Level Language Models
Lin Zheng, Vasilisa Bashlovkina, Timothy Dozat +3
Tokenizer-free language models eliminate the tokenizer step of the language modeling pipeline by operating directly on bytes; patch-based variants further aggregate contiguous byte…
ECLeKTic: a Novel Challenge Set for Evaluation of Cross-Lingual Knowledge Transfer
Omer Goldman, Uri Shaham, Dan Malkin +11
To achieve equitable performance across languages, large language models (LLMs) must be able to abstract knowledge beyond the language in which it was learnt. However, the current…
A Natural Bias for Language Generation Models
Clara Meister, Wojciech Stokowiec, Tiago Pimentel +3
After just a few hundred training updates, a standard probabilistic model for language generation has likely not yet learnt many semantic or syntactic rules of natural language, ma…
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Jack W. Rae, Sebastian Borgeaud, Trevor Cai +77
Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.…
Ethical and social risks of harm from Language Models
Laura Weidinger, John Mellor, Maribeth Rauh +20
This paper aims to help structure the risk landscape associated with large-scale Language Models (LMs). In order to foster advances in responsible innovation, an in-depth understan…
You should evaluate your language model on marginal likelihood over tokenisations
Kris Cao, Laura Rimell
Neural language models typically tokenise input text into sub-word units to achieve an open vocabulary. The standard approach is to use a single canonical tokenisation at both trai…
Pretraining the Noisy Channel Model for Task-Oriented Dialogue
Qi Liu, Lei Yu, Laura Rimell +1
Direct decoding for task-oriented dialogue is known to suffer from the explaining-away effect, manifested in models that prefer short and generic responses. Here we argue for the u…
Probing Emergent Semantics in Predictive Agents via Question Answering
Abhishek Das, Federico Carnevale, Hamza Merzic +8
Recent work has shown how predictive modeling can endow agents with rich knowledge of their surroundings, improving their ability to act in complex environments. We propose questio…
Syntactic Structure Distillation Pretraining For Bidirectional Encoders
Adhiguna Kuncoro, Lingpeng Kong, Daniel Fried +4
Textual representation learners trained on large amounts of data have achieved notable success on downstream tasks; intriguingly, they have also performed well on challenging tests…
Neural Generative Rhetorical Structure Parsing
Amandla Mabona, Laura Rimell, Stephen Clark +1
Rhetorical structure trees have been shown to be useful for several document-level tasks including summarization and document classification. Previous approaches to RST parsing hav…
Scalable Syntax-Aware Language Models Using Knowledge Distillation
Adhiguna Kuncoro, Chris Dyer, Laura Rimell +2
Prior work has shown that, on small amounts of training data, syntactic neural language models learn structurally sensitive generalisations more successfully than sequential langua…
Proceedings of the 2016 Workshop on Semantic Spaces at the Intersection of NLP, Physics and Cognitive Science
Dimitrios Kartsaklis, Martha Lewis, Laura Rimell
This volume contains the Proceedings of the 2016 Workshop on Semantic Spaces at the Intersection of NLP, Physics and Cognitive Science (SLPCS 2016), which was held on the 11th of J…
Take and Took, Gaggle and Goose, Book and Read: Evaluating the Utility of Vector Differences for Lexical Relation Learning
Ekaterina Vylomova, Laura Rimell, Trevor Cohn +1
Recent work on word embeddings has shown that simple vector subtraction over pre-trained embeddings is surprisingly effective at capturing different lexical relations, despite lack…
Using Sentence Plausibility to Learn the Semantics of Transitive Verbs
Tamara Polajnar, Laura Rimell, Stephen Clark
The functional approach to compositional distributional semantics considers transitive verbs to be linear maps that transform the distributional vectors representing nouns into a v…