70 citations · 357 across the 62 of their papers we have counts for
63 papers
MUSEKG: A Knowledge Graph Over Museum Collections
Jinhao Li, Jianzhong Qi, Soyeon Caren Han +1
Digitisation in the cultural heritage sector has produced large but fragmented repositories of museum collection data, spanning structured catalogue records, images, and unstructur…
MAP4TS: A Multi-Aspect Prompting Framework for Time-Series Forecasting with Large Language Models
Suchan Lee, Jihoon Choi, Sohyeon Lee +4
Recent advances have investigated the use of pretrained large language models (LLMs) for time-series forecasting by aligning numerical inputs with LLM embedding spaces. However, ex…
SynDoc: A Hybrid Discriminative-Generative Framework for Enhancing Synthetic Domain-Adaptive Document Key Information Extraction
Yihao Ding, Soyeon Caren Han, Yanbei Jiang +3
Domain-specific Visually Rich Document Understanding (VRDU) presents significant challenges due to the complexity and sensitivity of documents in fields such as medicine, finance,…
EMMM, Explain Me My Model! Explainable Machine Generated Text Detection in Dialogues
Angela Yifei Yuan, Haoyi Li, Soyeon Caren Han +1
The rapid adoption of large language models (LLMs) in customer service introduces new risks, as malicious actors can exploit them to conduct large-scale user impersonation through…
SPANER: Shared Prompt Aligner for Multimodal Semantic Representation
Thye Shan Ng, Caren Soyeon Han, Eun-Jung Holden
Recent advances in multimodal Parameter-Efficient Fine-Tuning (PEFT) have significantly improved performance on downstream tasks such as few-shot retrieval. However, most existing…
DocHop-QA: Towards Multi-Hop Reasoning over Multimodal Document Collections
Jiwon Park, Seohyun Pyeon, Jinwoo Kim +5
Despite rapid progress in large language models (LLMs), current QA benchmarks still overlook the core challenge of real-world scientific information seeking: synthesizing multimoda…
UniCast: A Unified Framework for Instance-Conditioned Multimodal Time-Series Forecasting
Sehyuk Park, Soyeon Caren Han, Eduard Hovy
Time series forecasting underpins applications in finance, healthcare, and environmental monitoring. Despite the success of Time Series Foundation Models (TSFMs), existing approach…
VRD-IU: Lessons from Visually Rich Document Intelligence and Understanding
Yihao Ding, Soyeon Caren Han, Yan Li +1
Visually Rich Document Understanding (VRDU) has emerged as a critical field in document intelligence, enabling automated extraction of key information from complex documents across…
MAGIC-VQA: Multimodal And Grounded Inference with Commonsense Knowledge for Visual Question Answering
Shuo Yang, Siwen Luo, Soyeon Caren Han +1
Visual Question Answering (VQA) requires reasoning across visual and textual modalities, yet Large Vision-Language Models (LVLMs) often lack integrated commonsense knowledge, limit…
SPADE: Structured Prompting Augmentation for Dialogue Enhancement in Machine-Generated Text Detection
Haoyi Li, Angela Yifei Yuan, Soyeon Caren Han +1
The increasing capability of large language models (LLMs) to generate synthetic content has heightened concerns about their misuse, driving the development of Machine-Generated Tex…
A Training-Free Length Extrapolation Approach for LLMs: Greedy Attention Logit Interpolation (GALI)
Yan Li, Tianyi Zhang, Zechuan Li +1
Transformer-based Large Language Models (LLMs) struggle with inputs exceeding their training context window due to positional out-of-distribution (O.O.D.) issues that disrupt atten…
Graph-Based Multimodal Contrastive Learning for Chart Question Answering
Yue Dai, Soyeon Caren Han, Wei Liu
Chart question answering (ChartQA) is challenged by the heterogeneous composition of chart elements and the subtle data patterns they encode. This work introduces a novel joint mul…
Multimodal Commonsense Knowledge Distillation for Visual Question Answering
Shuo Yang, Siwen Luo, Soyeon Caren Han
Existing Multimodal Large Language Models (MLLMs) and Visual Language Pretrained Models (VLPMs) have shown remarkable performances in the general Visual Question Answering (VQA). H…
TriG-NER: Triplet-Grid Framework for Discontinuous Named Entity Recognition
Rina Carines Cabral, Soyeon Caren Han, Areej Alhassan +3
Discontinuous Named Entity Recognition (DNER) presents a challenging problem where entities may be scattered across multiple non-adjacent tokens, making traditional sequence labell…
'No' Matters: Out-of-Distribution Detection in Multimodality Long Dialogue
Rena Gao, Xuetong Wu, Siwen Luo +2
Out-of-distribution (OOD) detection in multimodal contexts is essential for identifying deviations in combined inputs from different modalities, particularly in applications like o…
Multimodal Large Language Models and Tunings: Vision, Language, Sensors, Audio, and Beyond
Soyeon Caren Han, Feiqi Cao, Josiah Poon +1
This tutorial explores recent advancements in multimodal pretrained and large models, capable of integrating and processing diverse data forms such as text, images, audio, and vide…
SynJAC: Synthetic-data-driven Joint-granular Adaptation and Calibration for Domain Specific Scanned Document Key Information Extraction
Yihao Ding, Soyeon Caren Han, Zechuan Li +1
Visually Rich Documents (VRDs), comprising elements such as charts, tables, and paragraphs, convey complex information across diverse domains. However, extracting key information f…
MSG-Chart: Multimodal Scene Graph for ChartQA
Yue Dai, Soyeon Caren Han, Wei Liu
Automatic Chart Question Answering (ChartQA) is challenging due to the complex distribution of chart elements with patterns of the underlying data not explicitly displayed in chart…
3M-Health: Multimodal Multi-Teacher Knowledge Distillation for Mental Health Detection
Rina Carines Cabral, Siwen Luo, Josiah Poon +1
The significance of mental health classification is paramount in contemporary society, where digital platforms serve as crucial sources for monitoring individuals' well-being. Howe…
MIDAS: Multi-level Intent, Domain, And Slot Knowledge Distillation for Multi-turn NLU
Yan Li, So-Eon Kim, Seong-Bae Park +1
Although Large Language Models (LLMs) can generate coherent text, they often struggle to recognise user intent behind queries. In contrast, Natural Language Understanding (NLU) mod…
Deep Learning based Visually Rich Document Content Understanding: A Survey
Yihao Ding, Soyeon Caren Han, Jean Lee +1
Visually Rich Documents (VRDs) play a vital role in domains such as academia, finance, healthcare, and marketing, as they convey information through a combination of text, layout,…
3M: Multi-modal Multi-task Multi-teacher Learning for Game Event Detection
Thye Shan Ng, Feiqi Cao, Soyeon Caren Han
Esports has rapidly emerged as a global phenomenon with an ever-expanding audience via platforms, like YouTube. Due to the inherent complexity nature of the game, it is challenging…
Game-MUG: Multimodal Oriented Game Situation Understanding and Commentary Generation Dataset
Zhihao Zhang, Feiqi Cao, Yingbin Mo +3
The dynamic nature of esports makes the situation relatively complicated for average viewers. Esports broadcasting involves game expert casters, but the caster-dependent game comme…
PEACH: Pretrained-embedding Explanation Across Contextual and Hierarchical Structure
Feiqi Cao, Caren Han, Hyunsuk Chung
In this work, we propose a novel tree-based explanation technique, PEACH (Pretrained-embedding Explanation Across Contextual and Hierarchical Structure), that can explain how text-…