Automatic punctuation restoration with BERT models

Abstract

We present an approach for automatic punctuation restoration with BERT models for English and Hungarian. For English, we conduct our experiments on Ted Talks, a commonly used benchmark for punctuation restoration, while for Hungarian we evaluate our models on the Szeged Treebank dataset. Our best models achieve a macro-averaged $F_1$-score of 79.8 in English and 82.2 in Hungarian. Our code is publicly available.

11 pages, 6 figures, source code at https://github.com/attilanagy234/neural-punctuator

Cited by in corpus (4)

• Token-Level Supervised Contrastive Learning for Punctuation Restoration
• Streaming Punctuation: A Novel Punctuation Technique Leveraging Bidirectional Context for Continuous Speech Recognition
• Improving Punctuation Restoration for Speech Transcripts via External Data
• ViraPart: A Text Refinement Framework for Automatic Speech Recognition and Natural Language Processing Tasks in Persian