62 citations
Counting papers with at most 25 authors, so collaboration author lists do not stand in for the people here.
- University of CambridgeGB6 papers
- Tallinn University of TechnologyEE5 papers
- University of EdinburghGB5 papers
- University of AmsterdamNL4 papers
- Carnegie Mellon UniversityUS2 papers
- Tsinghua UniversityCN2 papers
- University of ChicagoUS2 papers
- Alibaba Group (United States)US1 paper
- Allen InstituteUS1 paper
- Amazon (United States)US1 paper
- Argonne National LaboratoryUS1 paper
- Artificial Intelligence in Medicine (Canada)CA1 paper
27 papers
Dialogue Is Not Enough to Make a Communicative BabyLM (But Neither Is Developmentally Inspired Reinforcement Learning)
Francesca Padovani, Bastian Bunzeck, Manar Ali +4
We investigate whether pre-training exclusively on dialogue data results in formally and functionally apt small language models. Based on this pre-trained llamalogue model, we empl…
Safe-SAIL: Towards a Fine-grained Safety Landscape of Large Language Models via Sparse Autoencoder Interpretation Framework
Jiaqi Weng, Han Zheng, Hanyu Zhang +6
Sparse autoencoders (SAEs) enable interpretability research by decomposing entangled model activations into monosemantic features. However, under what circumstances SAEs derive mos…
TurBLiMP: A Turkish Benchmark of Linguistic Minimal Pairs
Ezgi Başar, Francesca Padovani, Jaap Jumelet +1
We introduce TurBLiMP, the first Turkish benchmark of linguistic minimal pairs, designed to evaluate the linguistic abilities of monolingual and multilingual language models (LMs).…
Child-Directed Language Does Not Consistently Boost Syntax Learning in Language Models
Francesca Padovani, Jaap Jumelet, Yevgen Matusevych +1
Seminal work by Huebner et al. (2021) showed that language models (LMs) trained on English Child-Directed Language (CDL) can reach similar syntactic abilities as LMs trained on muc…
Multilingual Retrieval-Augmented Generation for Knowledge-Intensive Task
Leonardo Ranaldi, Barry Haddow, Alexandra Birch
Retrieval-augmented generation (RAG) has become a cornerstone of contemporary NLP, enhancing large language models (LLMs) by allowing them to access richer factual contexts through…
On the Consistency of Multilingual Context Utilization in Retrieval-Augmented Generation
Jirui Qi, Raquel Fernández, Arianna Bisazza
Retrieval-augmented generation (RAG) with large language models (LLMs) has demonstrated strong performance in multilingual question-answering (QA) tasks by leveraging relevant pass…
Workflows Community Summit 2024: Future Trends and Challenges in Scientific Workflows
Rafael Ferreira da Silva, Deborah Bard, Kyle Chard +108
The Workflows Community Summit gathered 111 participants from 18 countries to discuss emerging trends and challenges in scientific workflows, focusing on six key areas: time-sensit…
Faux Polyglot: A Study on Information Disparity in Multilingual Large Language Models
Nikhil Sharma, Kenton Murray, Ziang Xiao
Although the multilingual capability of LLMs offers new opportunities to overcome the language barrier, do these capabilities translate into real-life scenarios where linguistic di…
Model Internals-based Answer Attribution for Trustworthy Retrieval-Augmented Generation
Jirui Qi, Gabriele Sarti, Raquel Fernández +1
Ensuring the verifiability of model answers is a fundamental challenge for retrieval-augmented generation (RAG) in the question answering (QA) domain. Recently, self-citation promp…
HumanEval-XL: A Multilingual Code Generation Benchmark for Cross-lingual Natural Language Generalization
Qiwei Peng, Yekun Chai, Xuhong Li
Large language models (LLMs) have made significant progress in generating codes from textual prompts. However, existing benchmarks have mainly concentrated on translating English p…
A Multilingual Perspective on Probing Gender Bias
Karolina Stańczak
Gender bias represents a form of systematic negative treatment that targets individuals based on their gender. This discrimination can range from subtle sexist remarks and gendered…
On Bilingual Lexicon Induction with Large Language Models
Yaoyiran Li, Anna Korhonen, Ivan Vulić
Bilingual Lexicon Induction (BLI) is a core task in multilingual NLP that still, to a large extent, relies on calculating cross-lingual word representations. Inspired by the global…
Differential Privacy, Linguistic Fairness, and Training Data Influence: Impossibility and Possibility Theorems for Multilingual Language Models
Phillip Rust, Anders Søgaard
Language models such as mBERT, XLM-R, and BLOOM aim to achieve multilingual generalization or compression to facilitate transfer to a large number of (potentially unseen) languages…
Evaluating the Social Impact of Generative AI Systems in Systems and Society
Irene Solaiman, Zeerak Talat, William Agnew +28
Generative AI systems across modalities, ranging from text (including code), image, audio, and video, have broad social impacts, but there is no official standard for means of eval…
Translation-Enhanced Multilingual Text-to-Image Generation
Yaoyiran Li, Ching-Yun Chang, Stephen Rawls +2
Research on text-to-image generation (TTI) still predominantly focuses on the English language due to the lack of annotated image-caption data in other languages; in the long run,…
Are Pretrained Multilingual Models Equally Fair Across Languages?
Laura Cabello Piqueras, Anders Søgaard
Pretrained multilingual language models can help bridge the digital language divide, enabling high-quality NLP models for lower resourced languages. Studies of multilingual models…
Improving Bilingual Lexicon Induction with Cross-Encoder Reranking
Yaoyiran Li, Fangyu Liu, Ivan Vulić +1
Bilingual lexicon induction (BLI) with limited bilingual supervision is a crucial yet challenging task in multilingual NLP. Current state-of-the-art BLI methods rely on the inducti…
Conditioning and Sampling in Variational Diffusion Models for Speech Super-Resolution
Chin-Yun Yu, Sung-Lin Yeh, György Fazekas +1
Recently, diffusion models (DMs) have been increasingly used in audio processing tasks, including speech super-resolution (SR), which aims to restore high-frequency content given l…
Machine Reading, Fast and Slow: When Do Models "Understand" Language?
Sagnik Ray Choudhury, Anna Rogers, Isabelle Augenstein
Two of the most fundamental challenges in Natural Language Understanding (NLU) at present are: (a) how to establish whether deep learning-based models score highly on NLU benchmark…
Train Flat, Then Compress: Sharpness-Aware Minimization Learns More Compressible Models
Clara Na, Sanket Vaibhav Mehta, Emma Strubell
Model compression by way of parameter pruning, quantization, or distillation has recently gained popularity as an approach for reducing the computational requirements of modern dee…
Improving Word Translation via Two-Stage Contrastive Learning
Yaoyiran Li, Fangyu Liu, Nigel Collier +2
Word translation or bilingual lexicon induction (BLI) is a key cross-lingual task, aiming to bridge the lexical gap between different languages. In this work, we propose a robust a…
On Training Instance Selection for Few-Shot Neural Text Generation
Ernie Chang, Xiaoyu Shen, Hui-Syuan Yeh +1
Large-scale pretrained language models have led to dramatic improvements in text generation. Impressive performance can be achieved by finetuning only on a small number of instance…
Emergent Communication Pretraining for Few-Shot Machine Translation
Yaoyiran Li, Edoardo M. Ponti, Ivan Vulić +1
While state-of-the-art models that rely upon massively multilingual pretrained encoders achieve sample efficiency in downstream applications, they still require abundant amounts of…
Deep factorization for speech signal
Lantian Li, Dong Wang, Yixiang Chen +3
Various informative factors mixed in speech signals, leading to great difficulty when decoding any of the factors. An intuitive idea is to factorize each speech frame into individu…