984 citations · 3.3k across the 41 of their papers we have counts for
22 papers · 1 filter
Why transformers are obviously good models of language
Felix Hill
Nobody knows how language works, but many theories abound. Transformers are a class of neural networks that process language automatically with more success than alternatives, both…
Transformers generalize differently from information stored in context vs in weights
Stephanie C. Y. Chan, Ishita Dasgupta, Junkyung Kim +3
Transformer models can use two fundamentally different kinds of information: information stored in weights during training, and information provided ``in-context'' at inference tim…
Meaning without reference in large language models
Steven T. Piantadosi, Felix Hill
The widespread success of large language models (LLMs) has been met with skepticism that they possess anything like human concepts or meanings. Contrary to claims that LLMs possess…
Language models show human-like content effects on reasoning tasks
Ishita Dasgupta, Andrew K. Lampinen, Stephanie C. Y. Chan +5
Reasoning is a key ability for an intelligent system. Large language models (LMs) achieve above-chance performance on abstract reasoning tasks, but exhibit many imperfections. Howe…
Can language models learn from explanations in context?
Andrew K. Lampinen, Ishita Dasgupta, Stephanie C. Y. Chan +6
Language Models (LMs) can perform new tasks by adapting to a few in-context examples. For humans, explanations that connect examples to task principles can improve learning. We the…
BEAMetrics: A Benchmark for Language Generation Evaluation Evaluation
Thomas Scialom, Felix Hill
Natural language processing (NLP) systems are increasingly trained to generate open-ended text rather than classifying between responses. This makes research on evaluation metrics…
Grounded Language Learning Fast and Slow
Felix Hill, Olivier Tieleman, Tamara von Glehn +3
Recent work has shown that large text-based neural language models, trained with conventional supervised learning objectives, acquire a surprising propensity for few- and one-shot…
Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Text
Felix Hill, Sona Mokra, Nathaniel Wong +1
Recent work has described neural-network-based agents that are trained with reinforcement learning (RL) to execute language-like commands in simulated worlds, as a step towards an…
Extending Machine Language Models toward Human-Level Language Understanding
James L. McClelland, Felix Hill, Maja Rudolph +2
Language is crucial for human intelligence, but what exactly is its role? We take language to be a part of a system for understanding and communicating about situations. The human…
Higher-order Comparisons of Sentence Encoder Representations
Mostafa Abdou, Artur Kulmizev, Felix Hill +2
Representational Similarity Analysis (RSA) is a technique developed by neuroscientists for comparing activity patterns of different measurement modalities (e.g., fMRI, electrophysi…
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
Alex Wang, Yada Pruksachatkun, Nikita Nangia +5
In the last year, new models and methods for pretraining and transfer learning have driven striking performance improvements across a range of language understanding tasks. The GLU…
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Alex Wang, Amanpreet Singh, Julian Michael +3
For natural language understanding (NLU) technology to be maximally useful, both practically and as a scientific object of study, it must be general: it must be able to process lan…
Understanding Early Word Learning in Situated Artificial Agents
Felix Hill, Stephen Clark, Karl Moritz Hermann +1
Neural network-based systems can now learn to locate the referents of words and phrases in images, answer questions about visual scenes, and execute symbolic instructions as first-…
Grounded Language Learning in a Simulated 3D World
Karl Moritz Hermann, Felix Hill, Simon Green +11
We are increasingly surrounded by artificially intelligent technology that takes decisions and executes actions on our behalf. This creates a pressing need for general means to com…
SimVerb-3500: A Large-Scale Evaluation Set of Verb Similarity
Daniela Gerz, Ivan Vulić, Felix Hill +2
Verbs play a critical role in the meaning of sentences, but these ubiquitous words have received little attention in recent distributional semantics research. We introduce SimVerb-…
HyperLex: A Large-Scale Evaluation of Graded Lexical Entailment
Ivan Vulić, Daniela Gerz, Douwe Kiela +2
We introduce HyperLex - a dataset and evaluation resource that quantifies the extent of of the semantic category membership, that is, type-of relation also known as hyponymy-hypern…
Learning Distributed Representations of Sentences from Unlabelled Data
Felix Hill, Kyunghyun Cho, Anna Korhonen
Unsupervised methods for learning distributed representations of words are ubiquitous in today's NLP research, but far less is known about the best ways to learn distributed phrase…
The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
Felix Hill, Antoine Bordes, Sumit Chopra +1
We introduce a new test of how well language models capture meaning in children's books. Unlike standard language modelling benchmarks, it distinguishes the task of predicting synt…
Learning to Understand Phrases by Embedding the Dictionary
Felix Hill, Kyunghyun Cho, Anna Korhonen +1
Distributional models that learn rich semantic word representations are a success story of recent NLP research. However, developing models that learn useful representations of phra…
Embedding Word Similarity with Neural Machine Translation
Felix Hill, Kyunghyun Cho, Sebastien Jean +2
Neural language models learn word representations, or embeddings, that capture rich linguistic and conceptual information. Here we investigate the embeddings learned by neural mach…
Not All Neural Embeddings are Born Equal
Felix Hill, KyungHyun Cho, Sebastien Jean +2
Neural language models learn word representations that capture rich linguistic and conceptual information. Here we investigate the embeddings learned by neural machine translation…
SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimation
Felix Hill, Roi Reichart, Anna Korhonen
We present SimLex-999, a gold standard resource for evaluating distributional semantic models that improves on existing resources in several important ways. First, in contrast to g…