8 papers · 1 filter
Beyond Better Ideas: How Human-AI Collaboration Shapes Scientific Ideas and Human Perceptions
Qian Chen, Saatvik Pradhan, Janice Ahn +7
As AI increasingly becomes a collaborator in scientific research, its impact may extend beyond improving ideas to reshaping how they are evaluated, attributed, and perceived by hum…
CroissantMiner: Automated Extraction and Validation of Croissant Metadata for ML Datasets
Berke Arda, Ahmetcan Yavuz, Paul Gerry +7
Croissant has emerged as a standard for machine-readable dataset metadata, yet populating its fields remains labor-intensive and requires careful reading of accompanying dataset do…
New Evidence, Same Choice: Testing Physical Experiment Selection in Vision Language Models
Sourajit Saha, Shubhashis Roy Dipta, Nobin Sarwar +4
A model first sees an image from one physical measurement experiment, such as how far a block coasted, and must answer a question about a new trial, such as whether the block will…
BEFORE THE FLIP: Measuring Hidden Score Shifts In Quantized Vision Language Models Before The Answer Changes for Visual Question Answering
Sourajit Saha, Shubhashis Roy Dipta, Shaswati Saha +2
Quantization makes vision language models (VLMs) cheaper to store and run by using fewer bits to represent their weights. While unchanged answers on visual question answering (VQA)…
OracleZoom: On-Policy Self-Distillation Inspired Reference-Constrained Recursive Image Super Resolution
Shubhashis Roy Dipta, Sourajit Saha, Shaswati Saha +1
Recursive Super-Resolution (SR) extends fixed-scale SR to extreme magnification by repeatedly feeding predictions back into the same model, analogous to zooming an image repeatedly…
Last Translation Benchmark
Vilém Zouhar, Niyati Bafna, Mukund Choudhary +263
For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, stan…
Pluralis v0.1: Towards a Multicultural, Multimodal, Multilingual Benchmark for AI Risk and Reliability
Alicia Parrish, Rajat Shinde, Sanket Badhe +57
Current AI safety evaluation and benchmarking frameworks predominantly rely on Western-centric culture-agnostic defaults that mask critical regional laws, socio-linguistic nuances,…
Croissant Baker: Metadata Generation for Discoverable, Governable, and Reusable ML Datasets
Rafi Al Attrach, Rajna Fani, Sebastian Lobentanzer +17
Croissant has emerged as the metadata standard for machine learning datasets, providing a structured, JSON-LD-based format that makes dataset discovery, automated ingestion, and re…