243 citations · 246 across the 2 of their papers we have counts for
8 papers · 1 filter
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Jack W. Rae, Sebastian Borgeaud, Trevor Cai +77
Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.…
IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation
Samuel Cahyawijaya, Genta Indra Winata, Bryan Wilie +9
Natural language generation (NLG) benchmarks provide an important avenue to measure progress and develop better NLG systems. Unfortunately, the lack of publicly available NLG bench…
Mind the Gap: Assessing Temporal Generalization in Neural Language Models
Angeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya +11
Our world is open-ended, non-stationary, and constantly evolving; thus what we talk about and how we talk about it change over time. This inherent dynamic nature of language contra…
Syntactic Structure Distillation Pretraining For Bidirectional Encoders
Adhiguna Kuncoro, Lingpeng Kong, Daniel Fried +4
Textual representation learners trained on large amounts of data have achieved notable success on downstream tasks; intriguingly, they have also performed well on challenging tests…
Scalable Syntax-Aware Language Models Using Knowledge Distillation
Adhiguna Kuncoro, Chris Dyer, Laura Rimell +2
Prior work has shown that, on small amounts of training data, syntactic neural language models learn structurally sensitive generalisations more successfully than sequential langua…
Unsupervised Recurrent Neural Network Grammars
Yoon Kim, Alexander M. Rush, Lei Yu +3
Recurrent neural network grammars (RNNG) are generative models of language which jointly model syntax and surface structure by incrementally generating a syntax tree and sentence i…