Automatic Article Commenting: the Task and Dataset

Abstract

Comments of online articles provide extended views and improve user engagement. Automatically making comments thus become a valuable functionality for online forums, intelligent chatbots, etc. This paper proposes the new task of automatic article commenting, and introduces a large-scale Chinese dataset with millions of real comments and a human-annotated subset characterizing the comments' varying quality. Incorporating the human bias of comment quality, we further develop automatic metrics that generalize a broad set of popular reference-based metrics and exhibit greatly improved correlations with human evaluations.

arXiv commentACL2018; with supplements; Dataset link available in the paper

References in corpus (17)

• Adam: A Method for Stochastic Optimization
• Sequence to Sequence Learning with Neural Networks
• Microsoft COCO Captions: Data Collection and Evaluation Server
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation
• Neural Rating Regression with Abstractive Tips Generation for Recommendation
• Harnessing Deep Neural Networks with Logic Rules
• Relevance of Unsupervised Metrics in Task-Oriented Dialogue for Evaluating Natural Language Generation
• Toward Controlled Generation of Text
• deltaBLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse Targets
• On Unifying Deep Generative Models
• Recurrent Topic-Transition GAN for Visual Paragraph Generation
• Massive Exploration of Neural Machine Translation Architectures
• Context-aware Natural Language Generation with Recurrent Neural Networks
• From Eliza to XiaoIce: Challenges and Opportunities with Social Chatbots
• Challenges in Data-to-Document Generation
• Automated Crowdturfing Attacks and Defenses in Online Review Systems

Cited by in corpus (10)

• Coherent Comment Generation for Chinese Articles with a Graph-to-Sequence Model
• The Style-Content Duality of Attractiveness: Learning to Write Eye-Catching Headlines via Disentanglement
• Clickbait? Sensational Headline Generation with Auto-tuned Reinforcement Learning
• Towards Controlled and Diverse Generation of Article Comments
• Read, Attend and Comment: A Deep Architecture for Automatic News Comment Generation
• Generating Pertinent and Diversified Comments with Topic-aware Pointer-Generator Networks
• Chinese Pinyin Aided IME, Input What You Have Not Keystroked Yet
• Learning to Encode Evolutionary Knowledge for Automatic Commenting Long Novels
• DCA: Diversified Co-Attention towards Informative Live Video Commenting
• Automatic Generation of Personalized Comment Based on User Profile