17 citations · 34 across the 17 of their papers we have counts for
17 papers
Cross-lingual Self-Consistency for Multilingual Reasoning with Language Models
Ahmed Elhady, Eneko Agirre, Mikel Artetxe
Despite expanding their multilingual coverage, the advanced reasoning capabilities of LLMs remain largely confined to a few high-resource languages like English. To address this, w…
Revisiting Compositionality in Dual-Encoder Vision-Language Models: The Role of Inference
Imanol Miranda, Ander Salaberria, Eneko Agirre +1
Dual-encoder Vision-Language Models (VLMs) such as CLIP are often characterized as bag-of-words systems due to their poor performance on compositional benchmarks. We argue that thi…
Do not be greedy, Think Twice: Sampling and Selection for Document-level Information Extraction
Mikel Zubillaga, Oscar Sainz, Oier Lopez de Lacalle +1
Document-level Information Extraction (DocIE) aims to produce an output template with the entities, relations, and events of interest occurring in the given document. Standard prac…
Instructing Large Language Models for Low-Resource Languages: A Systematic Study for Basque
Oscar Sainz, Naiara Perez, Julen Etxaniz +9
Instructing language models with user intent requires large instruction datasets, which are only available for a limited set of languages. In this paper, we explore alternatives to…
Emergent Abilities of Large Language Models under Continued Pretraining for Language Adaptation
Ahmed Elhady, Eneko Agirre, Mikel Artetxe
Continued pretraining (CPT) is a popular approach to adapt existing large language models (LLMs) to new languages. When doing so, it is common practice to include a portion of Engl…
GuideX: Guided Synthetic Data Generation for Zero-Shot Information Extraction
Neil De La Fuente, Oscar Sainz, Iker García-Ferrero +1
Information Extraction (IE) systems are traditionally domain-specific, requiring costly adaptation that involves expert schema design, data annotation, and model training. While La…
WiCkeD: A Simple Method to Make Multiple Choice Benchmarks More Challenging
Ahmed Elhady, Eneko Agirre, Mikel Artetxe
We introduce WiCkeD, a simple method to increase the complexity of existing multiple-choice benchmarks by randomly replacing a choice with "None of the above", a method often used…
Data Contamination Report from the 2024 CONDA Shared Task
Oscar Sainz, Iker García-Ferrero, Alon Jacovi +25
The 1st Workshop on Data Contamination (CONDA 2024) focuses on all relevant aspects of data contamination in natural language processing, where data contamination is understood as…
Event Extraction in Basque: Typologically motivated Cross-Lingual Transfer-Learning Analysis
Mikel Zubillaga, Oscar Sainz, Ainara Estarrona +2
Cross-lingual transfer-learning is widely used in Event Extraction for low-resource languages and involves a Multilingual Language Model that is trained in a source language and ap…
Latxa: An Open Language Model and Evaluation Suite for Basque
Julen Etxaniz, Oscar Sainz, Naiara Perez +6
We introduce Latxa, a family of large language models for Basque ranging from 7 to 70 billion parameters. Latxa is based on Llama 2, which we continue pretraining on a new Basque c…
Improving Explicit Spatial Relationships in Text-to-Image Generation through an Automatically Derived Dataset
Ander Salaberria, Gorka Azkune, Oier Lopez de Lacalle +3
Existing work has observed that current text-to-image systems do not accurately reflect explicit spatial relations between objects such as 'left of' or 'below'. We hypothesize that…
PixT3: Pixel-based Table-To-Text Generation
Iñigo Alonso, Eneko Agirre, Mirella Lapata
Table-to-text generation involves generating appropriate textual descriptions given structured tabular data. It has attracted increasing attention in recent years thanks to the pop…
NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark
Oscar Sainz, Jon Ander Campos, Iker García-Ferrero +3
In this position paper, we argue that the classical evaluation on Natural Language Processing (NLP) tasks using annotated benchmarks is in trouble. The worst kind of data contamina…
Automatic Logical Forms improve fidelity in Table-to-Text generation
Iñigo Alonso, Eneko Agirre
Table-to-text systems generate natural language statements from structured data like tables. While end-to-end techniques suffer from low factual correctness (fidelity), a previous…
GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction
Oscar Sainz, Iker García-Ferrero, Rodrigo Agerri +3
Large Language Models (LLMs) combined with instruction tuning have made significant progress when generalizing to unseen tasks. However, they have been less successful in Informati…
CombLM: Adapting Black-Box Language Models through Small Fine-Tuned Models
Aitor Ormazabal, Mikel Artetxe, Eneko Agirre
Methods for adapting language models (LMs) to new tasks and domains have traditionally assumed white-box access to the model, and work by modifying its parameters. However, this is…
Label Verbalization and Entailment for Effective Zero- and Few-Shot Relation Extraction
Oscar Sainz, Oier Lopez de Lacalle, Gorka Labaka +2
Relation extraction systems require large amounts of labeled examples which are costly to annotate. In this work we reformulate relation extraction as an entailment task, with simp…