9 papers
CliniCARE-Bench: Clinical Calibrated Audit of Medical Reasoning in EHR
Veronica Chatrath, Bryan Zhu, George Pu +16
Large language models perform strongly on medical knowledge benchmarks, but reliable clinical deployment requires agents to conduct defensible investigations over heterogeneous, lo…
A Survey of Toxicity Detection and Mitigation Strategies for Multilingual Language Models
Soham Dan, Himanshu Beniwal, Thomas Hartvigsen
Large language models (LLMs) are increasingly deployed across languages, but their safety behavior remains uneven across linguistic and cultural contexts. This survey synthesizes w…
FindStatBench: Evaluating Large Language Models on Combinatorial Code Synthesis
Soham Dan
We introduce FindStatBench, an execution benchmark for evaluating large language models on combinatorial code synthesis. Built from FindStat, it contains 2,329 tasks across 24 coll…
LAUDE: LLM-Assisted Unit Test Generation and Debugging of Hardware DEsigns
Deeksha Nandal, Riccardo Revalor, Soham Dan +1
Unit tests are critical in the hardware design lifecycle to ensure that component design modules are functionally correct and conform to the specification before they are integrate…
Exploring the Robustness of Language Models for Tabular Question Answering via Attention Analysis
Kushal Raj Bhandari, Sixue Xing, Soham Dan +1
Large Language Models (LLMs), already shown to ace various unstructured text comprehension tasks, have also remarkably been shown to tackle table (structured) comprehension tasks w…
Large Language Model Confidence Estimation via Black-Box Access
Tejaswini Pedapati, Amit Dhurandhar, Soumya Ghosh +2
Estimating uncertainty or confidence in the responses of a model can be significant in evaluating trust not only in the responses, but also in the model as a whole. In this paper,…