150 citations · 178 across the 33 of their papers we have counts for
28 papers · 1 filter
Multimodal AI Detection for Math-Heavy Scientific Text
Yixuan Huang, Danish Pruthi, Bhuwan Dhingra
As large language models become increasingly capable of generating, and assisting with, scientific writing, there is a need for greater transparency and disclosure around the use o…
RegExtract: A Benchmark for PFAS Regulatory Information Extraction with LLMs
Defne Circi, Krystelle Lionti, Sarathkrishna Swaminathan +8
Extracting structured information from regulatory documents is a critical but challenging task. Relevant facts are scattered across long, legally dense texts, and traceability of e…
SearchAtlas: Analyzing Agentic Search Strategies via Evidential Query Graphs
Jiacheng Sang, Mengyuan Li, Sanxing Chen +3
LLM search agents are often evaluated on final-answer accuracy, overlooking the process. Analyzing a search strategy requires understanding how credible evidence is retrieved to ad…
Lazy Grounding: Attacking Search Agents with Factual Evidence
Yulin Zhang, Yukun Huang, Sanxing Chen +4
Search agents mitigate hallucination by grounding their answers in retrieved web results. However, retrieval-based approaches also introduce an attack surface: agents may cite misi…
Coding Agents are Effective Long-Context Processors
Weili Cao, Xunjian Yin, Bhuwan Dhingra +1
Large Language Models (LLMs) have demonstrated remarkable progress in scaling to access massive contexts. However, the access is via the latent and uninterpretable attention mechan…
InData: Towards Secure Multi-Step, Tool-Based Data Analysis
Karthikeyan K, Raghuveer Thirukovalluru, Bhuwan Dhingra +1
Large language model agents for data analysis typically generate and execute code directly on databases. However, when applied to sensitive data, this approach poses significant se…
A Platform for Investigating Public Health Content with Efficient Concern Classification
Christopher Li, Rickard Stureborg, Bhuwan Dhingra +1
A recent rise in online content expressing concerns with public health initiatives has contributed to already stalled uptake of preemptive measures globally. Future public health e…
MatViX: Multimodal Information Extraction from Visually Rich Articles
Ghazal Khalighinejad, Sharon Scott, Ollie Liu +4
Multimodal information extraction (MIE) is crucial for scientific literature, where valuable data is often spread across text, figures, and tables. In materials science, extracting…
To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contexts
Yukun Huang, Sanxing Chen, Hongyi Cai +1
Large Language Models (LLMs) are often augmented with external contexts, such as those used in retrieval-augmented generation (RAG). However, these contexts can be inaccurate or in…
Real-time Factuality Assessment from Adversarial Feedback
Sanxing Chen, Yukun Huang, Bhuwan Dhingra
We show that existing evaluations for assessing the factuality of news from conventional sources, such as claims on fact-checking websites, result in high accuracies over time for…
GenEOL: Harnessing the Generative Power of LLMs for Training-Free Sentence Embeddings
Raghuveer Thirukovalluru, Bhuwan Dhingra
Training-free embedding methods directly leverage pretrained large language models (LLMs) to embed text, bypassing the costly and complex procedure of contrastive learning. Previou…
Evaluating Morphological Compositional Generalization in Large Language Models
Mete Ismayilzada, Defne Circi, Jonne Sälevä +6
Large language models (LLMs) have demonstrated significant progress in various natural language generation and understanding tasks. However, their linguistic generalization capabil…
Atomic Self-Consistency for Better Long Form Generations
Raghuveer Thirukovalluru, Yukun Huang, Bhuwan Dhingra
Recent work has aimed to improve LLM generations by filtering out hallucinations, thereby improving the precision of the information in responses. Correctness of a long-form respon…
Tailoring Vaccine Messaging with Common-Ground Opinions
Rickard Stureborg, Sanxing Chen, Ruoyu Xie +6
One way to personalize chatbot interactions is by establishing common ground with the intended reader. A domain where establishing mutual understanding could be particularly impact…
ChatShop: Interactive Information Seeking with Language Agents
Sanxing Chen, Sam Wiseman, Bhuwan Dhingra
The desire and ability to seek new information strategically are fundamental to human learning but often overlooked in current language agent evaluation. We analyze a popular web s…
Extracting Polymer Nanocomposite Samples from Full-Length Documents
Ghazal Khalighinejad, Defne Circi, L. C. Brinson +1
This paper investigates the use of large language models (LLMs) for extracting sample lists of polymer nanocomposites (PNCs) from full-length materials science research papers. The…
Calibrating Long-form Generations from Large Language Models
Yukun Huang, Yixin Liu, Raghuveer Thirukovalluru +2
To enhance Large Language Models' (LLMs) reliability, calibration is essential -- the model's assessed confidence scores should align with the actual likelihood of its responses be…
Hierarchical Multi-Label Classification of Online Vaccine Concerns
Chloe Qinyu Zhu, Rickard Stureborg, Bhuwan Dhingra
Vaccine concerns are an ever-evolving target, and can shift quickly as seen during the COVID-19 pandemic. Identifying longitudinal trends in vaccine concerns and misinformation mig…
Hierarchical Multi-Instance Multi-Label Learning for Detecting Propaganda Techniques
Anni Chen, Bhuwan Dhingra
Since the introduction of the SemEval 2020 Task 11 (Martino et al., 2020a), several approaches have been proposed in the literature for classifying propaganda based on the rhetoric…
Selectively Answering Ambiguous Questions
Jeremy R. Cole, Michael J. Q. Zhang, Daniel Gillick +3
Trustworthy language models should abstain from answering questions when they do not know the answer. However, the answer to a question can be unknown for a variety of reasons. Pri…
Salient Span Masking for Temporal Understanding
Jeremy R. Cole, Aditi Chaudhary, Bhuwan Dhingra +1
Salient Span Masking (SSM) has shown itself to be an effective strategy to improve closed-book question answering performance. SSM extends general masked language model pretraining…
Learning the Legibility of Visual Text Perturbations
Dev Seth, Rickard Stureborg, Danish Pruthi +1
Many adversarial attacks in NLP perturb inputs to produce visually similar strings ('ergo' 'rgo') which are legible to humans but degrade model performance. Althou…
DIFFQG: Generating Questions to Summarize Factual Changes
Jeremy R. Cole, Palak Jain, Julian Martin Eisenschlos +3
Identifying the difference between two versions of the same article is useful to update knowledge bases and to understand how articles evolve. Paired texts occur naturally in diver…
On the State of the Art in Authorship Attribution and Authorship Verification
Jacob Tyo, Bhuwan Dhingra, Zachary C. Lipton
Despite decades of research on authorship attribution (AA) and authorship verification (AV), inconsistent dataset splits/filtering and mismatched evaluation methods make it difficu…