5 citations · 9 across the 12 of their papers we have counts for
13 papers
Word Boundary Information Isn't Useful for Encoder Language Models
Edward Gow-Smith, Dylan Phelps, Harish Tayyar Madabushi +2
All existing transformer-based approaches to NLP using subword tokenisation algorithms encode whitespace (word boundary information) through the use of special space symbols (such…
Evaluating Open-Domain Dialogues in Latent Space with Next Sentence Prediction and Mutual Information
Kun Zhao, Bohao Yang, Chenghua Lin +3
The long-standing one-to-many issue of the open-domain dialogues poses significant challenges for automatic evaluation methods, i.e., there may be multiple suitable responses which…
Assessing Linguistic Generalisation in Language Models: A Dataset for Brazilian Portuguese
Rodrigo Wilkens, Leonardo Zilio, Aline Villavicencio
Much recent effort has been devoted to creating large-scale language models. Nowadays, the most prominent approaches are based on deep neural networks, such as BERT. However, they…
Effective Cross-Task Transfer Learning for Explainable Natural Language Inference with T5
Irina Bigoulaeva, Rachneet Sachdeva, Harish Tayyar Madabushi +2
We compare sequential fine-tuning with a model for multi-task learning in the context where we are interested in boosting performance on two tasks, one of which depends on the othe…
SemEval-2022 Task 2: Multilingual Idiomaticity Detection and Sentence Embedding
Harish Tayyar Madabushi, Edward Gow-Smith, Marcos Garcia +3
This paper presents the shared task on Multilingual Idiomaticity Detection and Sentence Embedding, which consists of two subtasks: (a) a binary classification task aimed at identif…
Sample Efficient Approaches for Idiomaticity Detection
Dylan Phelps, Xuan-Rui Fan, Edward Gow-Smith +3
Deep neural models, in particular Transformer-based pre-trained language models, require a significant amount of data to train. This need for data tends to lead to problems when de…
Improving Tokenisation by Alternative Treatment of Spaces
Edward Gow-Smith, Harish Tayyar Madabushi, Carolina Scarton +1
Tokenisation is the first step in almost all NLP tasks, and state-of-the-art transformer-based language models all use subword tokenisation algorithms to process input text. Existi…
AStitchInLanguageModels: Dataset and Methods for the Exploration of Idiomaticity in Pre-Trained Language Models
Harish Tayyar Madabushi, Edward Gow-Smith, Carolina Scarton +1
Despite their success in a variety of NLP tasks, pre-trained language models, due to their heavy reliance on compositionality, fail in effectively capturing the meanings of multiwo…
Unsupervised Word Segmentation from Discrete Speech Units in Low-Resource Settings
Marcely Zanon Boito, Bolaji Yusuf, Lucas Ondel +2
Documenting languages helps to prevent the extinction of endangered dialects, many of which are otherwise expected to disappear by the end of the century. When documenting oral lan…
Investigating Language Impact in Bilingual Approaches for Computational Language Documentation
Marcely Zanon Boito, Aline Villavicencio, Laurent Besacier
For endangered languages, data collection campaigns have to accommodate the challenge that many of them are from oral tradition, and producing transcriptions is costly. Therefore,…
How Does Language Influence Documentation Workflow? Unsupervised Word Discovery Using Translations in Multiple Languages
Marcely Zanon Boito, Aline Villavicencio, Laurent Besacier
For language documentation initiatives, transcription is an expensive resource: one minute of audio is estimated to take one hour and a half on average of a linguist's work (Austin…
A small Griko-Italian speech translation corpus
Marcely Zanon Boito, Antonios Anastasopoulos, Marika Lekakou +2
This paper presents an extension to a very low-resource parallel corpus collected in an endangered language, Griko, making it useful for computational research. The corpus consists…
Matrix Factorization using Window Sampling and Negative Sampling for Improved Word Representations
Alexandre Salle, Marco Idiart, Aline Villavicencio
In this paper, we propose LexVec, a new method for generating distributed word representations that uses low-rank, weighted factorization of the Positive Point-wise Mutual Informat…