150 citations · 178 across the 33 of their papers we have counts for
14 papers · 1 filter
MatViX: Multimodal Information Extraction from Visually Rich Articles
Ghazal Khalighinejad, Sharon Scott, Ollie Liu +4
Multimodal information extraction (MIE) is crucial for scientific literature, where valuable data is often spread across text, figures, and tables. In materials science, extracting…
To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contexts
Yukun Huang, Sanxing Chen, Hongyi Cai +1
Large Language Models (LLMs) are often augmented with external contexts, such as those used in retrieval-augmented generation (RAG). However, these contexts can be inaccurate or in…
Real-time Factuality Assessment from Adversarial Feedback
Sanxing Chen, Yukun Huang, Bhuwan Dhingra
We show that existing evaluations for assessing the factuality of news from conventional sources, such as claims on fact-checking websites, result in high accuracies over time for…
GenEOL: Harnessing the Generative Power of LLMs for Training-Free Sentence Embeddings
Raghuveer Thirukovalluru, Bhuwan Dhingra
Training-free embedding methods directly leverage pretrained large language models (LLMs) to embed text, bypassing the costly and complex procedure of contrastive learning. Previou…
Evaluating Morphological Compositional Generalization in Large Language Models
Mete Ismayilzada, Defne Circi, Jonne Sälevä +6
Large language models (LLMs) have demonstrated significant progress in various natural language generation and understanding tasks. However, their linguistic generalization capabil…
Stratified Prediction-Powered Inference for Hybrid Language Model Evaluation
Adam Fisch, Joshua Maynez, R. Alex Hofer +3
Prediction-powered inference (PPI) is a method that improves statistical estimates based on limited human-labeled data. PPI achieves this by combining small amounts of human-labele…
Atomic Self-Consistency for Better Long Form Generations
Raghuveer Thirukovalluru, Yukun Huang, Bhuwan Dhingra
Recent work has aimed to improve LLM generations by filtering out hallucinations, thereby improving the precision of the information in responses. Correctness of a long-form respon…
Bayesian Prediction-Powered Inference
R. Alex Hofer, Joshua Maynez, Bhuwan Dhingra +3
Prediction-powered inference (PPI) is a method that improves statistical estimates based on limited human-labeled data. Specifically, PPI methods provide tighter confidence interva…
Tailoring Vaccine Messaging with Common-Ground Opinions
Rickard Stureborg, Sanxing Chen, Ruoyu Xie +6
One way to personalize chatbot interactions is by establishing common ground with the intended reader. A domain where establishing mutual understanding could be particularly impact…
ChatShop: Interactive Information Seeking with Language Agents
Sanxing Chen, Sam Wiseman, Bhuwan Dhingra
The desire and ability to seek new information strategically are fundamental to human learning but often overlooked in current language agent evaluation. We analyze a popular web s…
IsoBench: Benchmarking Multimodal Foundation Models on Isomorphic Representations
Deqing Fu, Ruohao Guo, Ghazal Khalighinejad +5
Current foundation models exhibit impressive capabilities when prompted either with text only or with both image and text inputs. But do their capabilities change depending on the…
Extracting Polymer Nanocomposite Samples from Full-Length Documents
Ghazal Khalighinejad, Defne Circi, L. C. Brinson +1
This paper investigates the use of large language models (LLMs) for extracting sample lists of polymer nanocomposites (PNCs) from full-length materials science research papers. The…
Calibrating Long-form Generations from Large Language Models
Yukun Huang, Yixin Liu, Raghuveer Thirukovalluru +2
To enhance Large Language Models' (LLMs) reliability, calibration is essential -- the model's assessed confidence scores should align with the actual likelihood of its responses be…
Hierarchical Multi-Label Classification of Online Vaccine Concerns
Chloe Qinyu Zhu, Rickard Stureborg, Bhuwan Dhingra
Vaccine concerns are an ever-evolving target, and can shift quickly as seen during the COVID-19 pandemic. Identifying longitudinal trends in vaccine concerns and misinformation mig…