36 citations · 69 across the 15 of their papers we have counts for
17 papers
Agentic Video Understanding: A Survey
Xinyu Deng, Siwen Luo, Daochang Liu
As large language models (LLMs) become capable of processing increasingly diverse modalities and longer temporal contexts, an emerging line of work is moving beyond fixed video-lan…
Locating Failure in Multi-Page Visually Rich Document Understanding Systems: An Empirical Attribution
Lewei Xu, Yihao Ding, Zihan Xu +5
Multi-page visually rich document understanding (MP-VRDU) systems can fail when evidence is not faithfully represented, not selected, or not used correctly, yet final accuracy alon…
Docs2Synth: A Synthetic Data Trained Retriever Framework for Scanned Visually Rich Documents Understanding
Yihao Ding, Qiang Sun, Puzhen Wu +3
Document understanding (VRDU) in regulated domains is particularly challenging, since scanned documents often contain sensitive, evolving, and domain specific knowledge. This leads…
A Survey on MLLM-based Visually Rich Document Understanding: Methods, Challenges, and Emerging Trends
Yihao Ding, Siwen Luo, Yue Dai +6
Visually Rich Document Understanding (VRDU) has become a pivotal area of research, driven by the need to automatically interpret documents that contain intricate visual, textual, a…
MAGIC-VQA: Multimodal And Grounded Inference with Commonsense Knowledge for Visual Question Answering
Shuo Yang, Siwen Luo, Soyeon Caren Han +1
Visual Question Answering (VQA) requires reasoning across visual and textual modalities, yet Large Vision-Language Models (LVLMs) often lack integrated commonsense knowledge, limit…
Multimodal Commonsense Knowledge Distillation for Visual Question Answering
Shuo Yang, Siwen Luo, Soyeon Caren Han
Existing Multimodal Large Language Models (MLLMs) and Visual Language Pretrained Models (VLPMs) have shown remarkable performances in the general Visual Question Answering (VQA). H…
3M-Health: Multimodal Multi-Teacher Knowledge Distillation for Mental Health Detection
Rina Carines Cabral, Siwen Luo, Josiah Poon +1
The significance of mental health classification is paramount in contemporary society, where digital platforms serve as crucial sources for monitoring individuals' well-being. Howe…
PDF-MVQA: A Dataset for Multimodal Information Retrieval in PDF-based Visual Question Answering
Yihao Ding, Kaixuan Ren, Jiabin Huang +2
Document Question Answering (QA) presents a challenge in understanding visually-rich documents (VRD), particularly those dominated by lengthy textual content like research journal…
Workshop on Document Intelligence Understanding
Soyeon Caren Han, Yihao Ding, Siwen Luo +5
Document understanding and information extraction include different tasks to understand a document and extract valuable information automatically. Recently, there has been a rising…
PDFVQA: A New Dataset for Real-World VQA on PDF Documents
Yihao Ding, Siwen Luo, Hyunsuk Chung +1
Document-based Visual Question Answering examines the document understanding of document images in conditions of natural language questions. We proposed a new document-based VQA da…
SceneGATE: Scene-Graph based co-Attention networks for TExt visual question answering
Feiqi Cao, Siwen Luo, Felipe Nunez +3
Most TextVQA approaches focus on the integration of objects, scene texts and question words by a simple transformer encoder. But this fails to capture the semantic relations betwee…
PiggyBack: Pretrained Visual Question Answering Environment for Backing up Non-deep Learning Professionals
Zhihao Zhang, Siwen Luo, Junyi Chen +4
We propose a PiggyBack, a Visual Question Answering platform that allows users to apply the state-of-the-art visual-language pretrained models easily. The PiggyBack supports the fu…
Doc-GCN: Heterogeneous Graph Convolutional Networks for Document Layout Analysis
Siwen Luo, Yihao Ding, Siqu Long +2
Recognizing the layout of unstructured digital documents is crucial when parsing the documents into the structured, machine-readable format for downstream applications. Recent stud…
Local Interpretations for Explainable Natural Language Processing: A Survey
Siwen Luo, Hamish Ivison, Caren Han +1
As the use of deep learning techniques has grown across various fields over the past decade, complaints about the opaqueness of the black-box models have increased, resulting in an…
Deep Structured Feature Networks for Table Detection and Tabular Data Extraction from Scanned Financial Document Images
Siwen Luo, Mengting Wu, Yiwen Gong +2
Automatic table detection in PDF documents has achieved a great success but tabular data extraction are still challenging due to the integrity and noise issues in detected table ar…
VICTR: Visual Information Captured Text Representation for Text-to-Image Multimodal Tasks
Soyeon Caren Han, Siqu Long, Siwen Luo +2
Text-to-image multimodal tasks, generating/retrieving an image from a given text description, are extremely challenging tasks since raw text descriptions cover quite limited inform…
REXUP: I REason, I EXtract, I UPdate with Structured Compositional Reasoning for Visual Question Answering
Siwen Luo, Soyeon Caren Han, Kaiyuan Sun +1
Visual question answering (VQA) is a challenging multi-modal task that requires not only the semantic understanding of both images and questions, but also the sound perception of a…