6 papers
MULTISEISMO: A Multimodal Seismic Dataset and Model for Cross-Modal Seismic Understanding
Sai Munikoti, Ian Stewart, Chengping Chai +4
The application of generalist multimodal models (GMMs) to specialized scientific domains remains limited due to the scarcity of comprehensive domain-specific datasets that integrat…
Back to the Barn with LLAMAs: Evolving Pretrained LLM Backbones in Finetuning Vision Language Models
Sameera Horawalavithana, Lauren Phillips, Ian Stewart +2
Vision-Language Models (VLMs) have rapidly advanced by leveraging powerful pre-trained Large Language Models (LLMs) as core reasoning backbones. As new and more capable LLMs emerge…
Directional Concentration Uncertainty: A representational approach to uncertainty quantification for generative models
Souradeep Chattopadhyay, Brendan Kennedy, Sai Munikoti +2
In the critical task of making generative models trustworthy and robust, methods for Uncertainty Quantification (UQ) have begun to show encouraging potential. However, many of thes…
Uncertainty Quantification for Named Entity Recognition via Full-Sequence and Subsequence Conformal Prediction
Matthew Singer, Srijan Sengupta, Karl Pazdernik
Named Entity Recognition (NER) serves as a foundational component in many natural language processing (NLP) pipelines. However, current NER models typically output a single predict…
Surprisingly Fragile: Assessing and Addressing Prompt Instability in Multimodal Foundation Models
Ian Stewart, Sameera Horawalavithana, Brendan Kennedy +2
Multimodal foundation models (MFMs) such as OFASys show the potential to unlock analysis of complex data such as images, videos, and audio data via text prompts alone. However, the…
Benchmarking LLMs for Environmental Review and Permitting
Rounak Meyur, Hung Phan, Koby Hayashi +12
The National Environment Policy Act (NEPA) stands as a foundational piece of environmental legislation in the United States, requiring federal agencies to consider the environmenta…