Advancing NLP with Cognitive Language Processing Signals

Abstract

When we read, our brain processes language and generates cognitive processing data such as gaze patterns and brain activity. These signals can be recorded while reading. Cognitive language processing data such as eye-tracking features have shown improvements on single NLP tasks. We analyze whether using such human features can show consistent improvement across tasks and data sources. We present an extensive investigation of the benefits and limitations of using cognitive processing data for NLP. Specifically, we use gaze and EEG features to augment models of named entity recognition, relation classification, and sentiment analysis. These methods significantly outperform the baselines and show the potential and current limitations of employing human language processing data for NLP.

References in corpus (1)

• Entity Recognition at First Sight: Improving NER with Eye Movement Information

Cited by in corpus (7)

• Decoding EEG Brain Activity for Multi-Modal Natural Language Processing
• Improving Natural Language Processing Tasks with Human Gaze-Guided Neural Attention
• Utilizing Cognitive Signals Generated during Human Reading to Enhance Keyphrase Extraction from Microblogs
• Human brain activity for machine attention
• OASST-ETC Dataset: Alignment Signals from Eye-tracking Analysis of LLM Responses
• CogAlign: Learning to Align Textual Neural Representations to Cognitive Language Processing Signals
• Leveraging recent advances in Pre-Trained Language Models forEye-Tracking Prediction