A BERT Baseline for the Natural Questions

Abstract

This technical note describes a new baseline for the Natural Questions. Our model is based on BERT and reduces the gap between the model F1 scores reported in the original dataset paper and the human upper bound by 30% and 50% relative for the long and short answer tasks respectively. This baseline has been submitted to the official NQ leaderboard at ai.google.com/research/NaturalQuestions. Code, preprocessed data and pretrained model are available at https://github.com/google-research/language/tree/master/language/question_answering/bert_joint.

Cited by in corpus (36)

• Big Bird: Transformers for Longer Sequences
• Pretrained Transformers for Text Ranking: BERT and Beyond
• Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering
• On Identifiability in Transformers
• What do Models Learn from Question Answering Datasets?
• TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages
• ETC: Encoding Long and Structured Inputs in Transformers
• EntQA: Entity Linking as Question Answering
• Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering
• Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding
• End-to-End QA on COVID-19: Domain Adaptation with Synthetic Training
• A Discrete Hard EM Approach for Weakly Supervised Question Answering
• Frustratingly Easy Natural Question Answering
• RikiNet: Reading Wikipedia Pages for Natural Question Answering
• Synthetic QA Corpora Generation with Roundtrip Consistency
• GLGE: A New General Language Generation Evaluation Benchmark
• Challenges in Information-Seeking QA: Unanswerable Questions and Paragraph Retrieval
• CFO: A Framework for Building Production NLP Systems
• Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension
• Can NLI Models Verify QA Systems' Predictions?
• ReCO: A Large Scale Chinese Reading Comprehension Dataset on Opinion
• Summary-Oriented Question Generation for Informational Queries
• Android Security using NLP Techniques: A Review
• BERT-based knowledge extraction method of unstructured domain text
• Towards Confident Machine Reading Comprehension
• ActBERT: Learning Global-Local Video-Text Representations
• An Exploration of Data Augmentation and Sampling Techniques for Domain-Agnostic Question Answering
• VAULT: VAriable Unified Long Text Representation for Machine Reading Comprehension
• When to Fold'em: How to answer Unanswerable questions
• Toward Deconfounding the Influence of Entity Demographics for Question Answering Accuracy
• What's in a Name? Answer Equivalence For Open-Domain Question Answering
• Investigating Post-pretraining Representation Alignment for Cross-Lingual Question Answering
• A question-answering system for aircraft pilots' documentation
• No Answer is Better Than Wrong Answer: A Reflection Model for Document Level Machine Reading Comprehension
• Multi-Granularity Representations of Dialog
• Knowing More About Questions Can Help: Improving Calibration in Question Answering