4 papers
Last Translation Benchmark
Vilém Zouhar, Niyati Bafna, Mukund Choudhary +263
For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, stan…
Benchmarking Large Language Models for Knowledge Graph Validation
Farzad Shami, Stefano Marchesin, Gianmaria Silvello
Knowledge Graphs (KGs) store structured factual knowledge by linking entities through relationships, crucial for many applications. These applications depend on the KG's factual ac…
From Single to Multi-Agent Reasoning: Advancing GeneGPT for Genomics QA
Kimia Abedini, Farzad Shami, Gianmaria Silvello
Comprehending genomic information is essential for biomedical research, yet extracting data from complex distributed databases remains challenging. Large language models (LLMs) off…
GROKE: Vision-Free Navigation Instruction Evaluation via Graph Reasoning on OpenStreetMap
Farzad Shami, Subhrasankha Dey, Nico Van de Weghe +1
The evaluation of navigation instructions remains a persistent challenge in Vision-and-Language Navigation (VLN) research. Traditional reference-based metrics such as BLEU and ROUG…