ConceptEVA: Concept-Based Interactive Exploration and Customization of Document Summaries

Abstract

With the most advanced natural language processing and artificial intelligence approaches, effective summarization of long and multi-topic documents -- such as academic papers -- for readers from different domains still remains a challenge. To address this, we introduce ConceptEVA, a mixed-initiative approach to generate, evaluate, and customize summaries for long and multi-topic documents. ConceptEVA incorporates a custom multi-task longformer encoder decoder to summarize longer documents. Interactive visualizations of document concepts as a network reflecting both semantic relatedness and co-occurrence help users focus on concepts of interest. The user can select these concepts and automatically update the summary to emphasize them. We present two iterations of ConceptEVA evaluated through an expert review and a within-subjects study. We find that participants' satisfaction with customized summaries through ConceptEVA is higher than their own manually-generated summary, while incorporating critique into the summaries proved challenging. Based on our findings, we make recommendations for designing summarization systems incorporating mixed-initiative interactions.

arXiv comment16 pages, 7 figures

References in corpus (11)

• UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
• BERTScore: Evaluating Text Generation with BERT
• Longformer: The Long-Document Transformer
• PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization
• Improving language models by retrieving from trillions of tokens
• Embedding Projector: Interactive Visualization and Interpretation of Embeddings
• Recursively Summarizing Books with Human Feedback
• VitaLITy: Promoting Serendipitous Discovery of Academic Literature with Transformers & Visual Analytics
• ConceptScope: Organizing and Visualizing Knowledge in Documents based on Domain Ontology
• A preliminary study on evaluating Consultation Notes with Post-Editing
• Meta Learning for Code Summarization

Cited by in corpus (3)

• GazeNoter: Co-Piloted AR Note-Taking via Gaze Selection of LLM Suggestions to Match Users' Intentions
• RealitySummary: Exploring On-Demand Mixed Reality Text Summarization and Question Answering using Large Language Models
• Visual Text Mining with Progressive Taxonomy Construction for Environmental Studies