2 citations · 5 across the 19 of their papers we have counts for
6 papers · 1 filter
MetaMetrics-MT: Tuning Meta-Metrics for Machine Translation via Human Preference Calibration
David Anugraha, Garry Kuwanto, Lucky Susanto +2
We present MetaMetrics-MT, an innovative metric designed to evaluate machine translation (MT) tasks by aligning closely with human preferences through Bayesian optimization with Ga…
WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines
Genta Indra Winata, Frederikus Hudi, Patrick Amadeus Irawan +48
Vision Language Models (VLMs) often struggle with culture-specific knowledge, particularly in languages other than English and in underrepresented cultural contexts. To evaluate th…
MetaMetrics: Calibrating Metrics For Generation Tasks Using Human Preferences
Genta Indra Winata, David Anugraha, Lucky Susanto +2
Understanding the quality of a performance evaluation metric is crucial for ensuring that model outputs align with human preferences. However, it remains unclear how well each metr…
URIEL+: Enhancing Linguistic Inclusion and Usability in a Typological and Multilingual Knowledge Base
Aditya Khan, Mason Shipton, David Anugraha +5
URIEL is a knowledge base offering geographical, phylogenetic, and typological vector representations for 7970 languages. It includes distance measures between these vectors for 40…
ProxyLM: Predicting Language Model Performance on Multilingual Tasks via Proxy Models
David Anugraha, Genta Indra Winata, Chenyue Li +2
Performance prediction is a method to estimate the performance of Language Models (LMs) on various Natural Language Processing (NLP) tasks, mitigating computational costs associate…
Predicting Machine Translation Performance on Low-Resource Languages: The Role of Domain Similarity
Eric Khiu, Hasti Toossi, David Anugraha +6
Fine-tuning and testing a multilingual large language model is expensive and challenging for low-resource languages (LRLs). While previous studies have predicted the performance of…