A Hybrid Intelligence Method for Argument Mining
arXiv:2403.09713 · doi:10.1613/jair.1.15135
Abstract
Large-scale survey tools enable the collection of citizen feedback in opinion corpora. Extracting the key arguments from a large and noisy set of opinions helps in understanding the opinions quickly and accurately. Fully automated methods can extract arguments but (1) require large labeled datasets that induce large annotation costs and (2) work well for known viewpoints, but not for novel points of view. We propose HyEnA, a hybrid (human + AI) method for extracting arguments from opinionated texts, combining the speed of automated processing with the understanding and reasoning capabilities of humans. We evaluate HyEnA on three citizen feedback corpora. We find that, on the one hand, HyEnA achieves higher coverage and precision than a state-of-the-art automated method when compared to a common set of diverse opinions, justifying the need for human insight. On the other hand, HyEnA requires less human effort and does not compromise quality compared to (fully manual) expert analysis, demonstrating the benefit of combining human and artificial intelligence.
Published in JAIR
References in corpus (15)
- Fast unfolding of communities in large networks
- Survey of Hallucination in Natural Language Generation
- Training language models to follow instructions with human feedback
- LLaMA: Open and Efficient Foundation Language Models
- BERTScore: Evaluating Text Generation with BERT
- BERTopic: Neural topic modeling with a class-based TF-IDF procedure
- Fine-tuning language models to find agreement among humans with diverse preferences
- On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective
- What is the Essence of a Claim? Cross-Domain Claim Identification
- Reason Against the Machine: Future Directions for Mass Online Deliberation
- Cross-lingual Argumentation Mining: Machine Translation (and a bit of Projection) is All You Need!
- Do Differences in Values Influence Disagreements in Online Discussions?
- From Key Points to Key Point Hierarchy: Structured and Expressive Opinion Summarization
- An Empirical Analysis of Diversity in Argument Summarization
- Annotator-Centric Active Learning for Subjective NLP Tasks