4 papers
Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks
Benjamin Warner, Ratna Sagari Grandhi, Max Kieffer +32
Evaluating large language models (LLMs) for medical applications remains challenging due to benchmark saturation, limited data accessibility, and insufficient coverage of relevant…
Where Do Vision-Language Models Fail? World Scale Analysis for Image Geolocalization
Siddhant Bharadwaj, Ashish Vashist, Fahimul Aleem +1
Image geolocalization has traditionally been addressed through retrieval-based place recognition or geometry-based visual localization pipelines. Recent advances in Vision-Language…
Beyond Accuracy: Evaluating Visual Grounding In Multimodal Medical Reasoning
Anas Zafar, Leema Krishna Murali, Ashish Vashist
Recent work shows that text-only reinforcement learning with verifiable rewards (RLVR) can match or outperform image-text RLVR on multimodal medical VQA benchmarks, suggesting curr…
FOD-S2R: A FOD Dataset for Sim2Real Transfer Learning based Object Detection
Ashish Vashist, Qiranul Saadiyean, Suresh Sundaram +1
Foreign Object Debris (FOD) within aircraft fuel tanks presents critical safety hazards including fuel contamination, system malfunctions, and increased maintenance costs. Despite…