4 papers · 1 filter
Benchmarking the Generality of Vision-Language-Action Models
Pranav Guruprasad, Sudipta Chowdhury, Harsh Sikka +6
Generalist multimodal agents are expected to unify perception, language, and control - operating robustly across diverse real world domains. However, current evaluation practices r…
Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
Tyler A. Chang, Catherine Arnett, Abdelrahman Sadallah +377
To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we pre…
Confirmation bias: A challenge for scalable oversight
Gabriel Recchia, Chatrik Singh Mangat, Jinu Nyachhyon +4
Scalable oversight protocols aim to empower evaluators to accurately verify AI models more capable than themselves. However, human evaluators are subject to biases that can lead to…
Local Herb Identification Using Transfer Learning: A CNN-Powered Mobile Application for Nepalese Flora
Prajwal Thapa, Mridul Sharma, Jinu Nyachhyon +1
Herb classification presents a critical challenge in botanical research, particularly in regions with rich biodiversity such as Nepal. This study introduces a novel deep learning a…