5 papers
DisasterBench: Benchmarking LLM Planning under Typed Tool Interface Constraints
Zhitong Chen, Kai Yin, Weifeng Zhang +7
Disasters cause severe societal impacts, demanding rapid coordination of heterogeneous AI tools, from satellite analysis to flood prediction and damage assessment, into coherent mu…
DisastQA: A Comprehensive Benchmark for Evaluating Question Answering in Disaster Management
Zhitong Chen, Kai Yin, Xiangjue Dong +7
Accurate question answering (QA) in disaster management requires reasoning over uncertain and conflicting information, a setting poorly captured by existing benchmarks built on cle…
DMRetriever: A Family of Models for Improved Text Retrieval in Disaster Management
Kai Yin, Xiangjue Dong, Chengkai Liu +4
Effective and efficient access to relevant information is essential for disaster management. However, no retrieval model is specialized for disaster management, and existing genera…
DisastIR: A Comprehensive Information Retrieval Benchmark for Disaster Management
Kai Yin, Xiangjue Dong, Chengkai Liu +5
Effective disaster management requires timely access to accurate and contextually relevant information. Existing Information Retrieval (IR) benchmarks, however, focus primarily on…
CrisisSense-LLM: Instruction Fine-Tuned Large Language Model for Multi-label Social Media Text Classification in Disaster Informatics
Kai Yin, Bo Li, Chengkai Liu +2
In the field of crisis/disaster informatics, social media is increasingly being used for improving situational awareness to inform response and relief efforts. Efficient and accura…